2023-07-14T18:57:10.948 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T18:57:10.950 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053 --verbose --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} --owner scheduled_pdonnell@teuthology --name pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/orig.config.yaml 2023-07-14T18:57:11.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T18:57:11.100 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053 branch: wip-pdonnell-testing-20230714.134335 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7338053' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20230714.134335 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20230714.134335 sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5179 sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec sleep_before_teardown: 0 subset: 285/512 suite: fs:workload suite_branch: wip-pdonnell-testing-20230714.134335 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi133.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: aa6a80e6cc655fff48452a7eca02070e61d074fc timestamp: 2023-07-14_16:44:47 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-14T18:57:11.101 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa; will attempt to use it 2023-07-14T18:57:11.102 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks 2023-07-14T18:57:11.102 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-14T18:57:11.109 INFO:teuthology.task.internal:Checking packages... 2023-07-14T18:57:11.144 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81c9387d2e7576da065747ec9c16d1c7d69016ec' 2023-07-14T18:57:11.144 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-14T18:57:11.145 INFO:teuthology.packaging:ref: None 2023-07-14T18:57:11.145 INFO:teuthology.packaging:tag: None 2023-07-14T18:57:11.146 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20230714.134335 2023-07-14T18:57:11.146 INFO:teuthology.packaging:sha1: 81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T18:57:11.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20230714.134335 2023-07-14T18:57:11.447 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4940.g81c9387d 2023-07-14T18:57:11.448 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-14T18:57:11.455 INFO:teuthology.task.internal:no buildpackages task found 2023-07-14T18:57:11.455 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-14T18:57:11.523 INFO:teuthology.task.internal:Saving configuration 2023-07-14T18:57:11.558 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-14T18:57:11.565 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T18:57:11.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:01.891886', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T18:57:11.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:01.893602', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T18:57:11.653 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:01.894798', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T18:57:11.654 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-14T18:57:11.660 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-14T18:57:11.660 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-14T18:57:11.660 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-14T18:57:11.661 INFO:teuthology.run_tasks:Running task console_log... 2023-07-14T18:57:11.789 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f009b3d6430>, signals=[15]) 2023-07-14T18:57:11.790 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-14T18:57:11.797 INFO:teuthology.task.internal:Opening connections... 2023-07-14T18:57:11.797 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T18:57:11.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:57:11.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T18:57:11.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:57:11.939 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T18:57:11.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:57:12.010 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-14T18:57:12.016 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2023-07-14T18:57:12.039 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2023-07-14T18:57:12.039 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2023-07-14T18:57:12.096 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:57:12.097 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:57:12.097 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2023-07-14T18:57:12.097 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2023-07-14T18:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.6" 2023-07-14T18:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2023-07-14T18:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:57:12.098 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:57:12.099 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:57:12.099 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:57:12.099 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T18:57:12.099 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:57:12.099 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:57:12.100 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:57:12.100 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:57:12.101 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2023-07-14T18:57:12.125 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-07-14T18:57:12.144 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-07-14T18:57:12.145 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-07-14T18:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout:ID="rhel" 2023-07-14T18:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="fedora" 2023-07-14T18:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8.6" 2023-07-14T18:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:57:12.204 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:57:12.204 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-07-14T18:57:12.204 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:57:12.204 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:57:12.204 INFO:teuthology.orchestra.run.smithi133.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:57:12.205 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:57:12.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T18:57:12.205 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:57:12.205 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:57:12.205 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:57:12.206 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:57:12.206 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-07-14T18:57:12.235 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2023-07-14T18:57:12.254 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2023-07-14T18:57:12.255 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-07-14T18:57:12.314 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:57:12.315 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:57:12.316 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:57:12.317 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:57:12.317 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:57:12.317 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-07-14T18:57:12.342 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-14T18:57:12.355 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-14T18:57:12.393 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-14T18:57:12.393 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2023-07-14T18:57:12.396 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2023-07-14T18:57:12.398 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2023-07-14T18:57:12.417 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-14T18:57:12.424 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-14T18:57:12.424 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2023-07-14T18:57:12.454 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2023-07-14T18:57:12.457 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2023-07-14T18:57:12.485 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-14T18:57:17.805 INFO:teuthology.run_tasks:Running task kernel... 2023-07-14T18:57:17.823 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-07-14T18:57:17.824 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-07-14T18:57:17.824 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-07-14T18:57:17.825 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T18:57:17.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:57:17.846 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2023-07-14T18:57:17.906 INFO:teuthology.orchestra.run.smithi082.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-14T18:57:17.906 INFO:teuthology.task.kernel:Running kernel on smithi082: 4.18.0-372.9.1.el8.x86_64 2023-07-14T18:57:17.907 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2023-07-14T18:57:19.148 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T18:57:19.149 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-07-14T18:57:20.867 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 7.8 kB/s | 3.5 kB 00:00 2023-07-14T18:57:23.943 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 16 MB 00:03 2023-07-14T18:57:27.409 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 421 kB/s | 24 kB 00:00 2023-07-14T18:57:28.086 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-07-14T18:57:28.134 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-07-14T18:57:28.135 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-07-14T18:57:28.136 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-07-14T18:57:28.255 DEBUG:teuthology.orchestra.run.smithi082:> echo no | sudo yum reinstall kernel || true 2023-07-14T18:57:28.629 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T18:57:28.629 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-07-14T18:57:28.898 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:01 ago on Fri 14 Jul 2023 06:57:27 PM UTC. 2023-07-14T18:57:28.938 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-14T18:57:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-07-14T18:57:29.023 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum reinstall -y kernel || true 2023-07-14T18:57:29.351 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T18:57:29.351 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-07-14T18:57:29.611 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Fri 14 Jul 2023 06:57:27 PM UTC. 2023-07-14T18:57:29.651 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-07-14T18:57:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-07-14T18:57:29.737 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2023-07-14T18:57:29.817 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-07-14T18:57:29.817 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-07-14T18:57:29.817 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-07-14T18:57:29.818 DEBUG:teuthology.parallel:result is None 2023-07-14T18:57:29.818 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-14T18:57:29.825 INFO:teuthology.task.internal:Creating test directory... 2023-07-14T18:57:29.825 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T18:57:29.827 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T18:57:29.830 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-14T18:57:29.850 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-14T18:57:29.857 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-14T18:57:29.887 INFO:teuthology.task.internal:Creating archive directory... 2023-07-14T18:57:29.887 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T18:57:29.890 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T18:57:29.892 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-14T18:57:30.010 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-14T18:57:30.017 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-14T18:57:30.017 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T18:57:30.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:57:30.036 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-14T18:57:30.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:57:30.055 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T18:57:30.057 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T18:57:30.078 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-14T18:57:30.092 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T18:57:30.108 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T18:57:30.123 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T18:57:30.135 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T18:57:30.149 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T18:57:30.150 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-14T18:57:30.152 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-14T18:57:30.159 INFO:teuthology.task.internal:Configuring sudo... 2023-07-14T18:57:30.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T18:57:30.166 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T18:57:30.168 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-14T18:57:30.220 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-14T18:57:30.229 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-14T18:57:30.230 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T18:57:30.232 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T18:57:30.237 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-14T18:57:30.277 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T18:57:33.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T18:57:33.068 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T18:57:33.117 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T18:57:33.159 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T18:57:33.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T18:57:33.229 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T18:57:33.279 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T18:57:33.484 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T18:57:33.534 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T18:57:33.578 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T18:57:33.578 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T18:57:33.649 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T18:57:38.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-14T18:57:38.760 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T18:57:38.804 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-14T18:57:38.852 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T18:57:38.852 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-14T18:57:38.922 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2023-07-14T18:57:38.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2023-07-14T18:57:38.927 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2023-07-14T18:57:38.993 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T18:57:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T18:57:39.027 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T18:57:39.450 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-14T18:57:39.457 INFO:teuthology.task.internal:Starting timer... 2023-07-14T18:57:39.457 INFO:teuthology.run_tasks:Running task pcp... 2023-07-14T18:57:39.489 INFO:teuthology.run_tasks:Running task selinux... 2023-07-14T18:57:39.521 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-07-14T18:57:39.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2023-07-14T18:57:39.574 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2023-07-14T18:57:39.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2023-07-14T18:57:39.638 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2023-07-14T18:57:39.638 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2023-07-14T18:57:39.708 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2023-07-14T18:57:39.710 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-14T18:57:39.711 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2023-07-14T18:57:39.739 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2023-07-14T18:57:39.739 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2023-07-14T18:57:39.761 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2023-07-14T18:57:39.761 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2023-07-14T18:57:39.784 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2023-07-14T18:57:39.785 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2023-07-14T18:57:39.785 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T18:57:39.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:57:39.813 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T18:57:39.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:57:39.841 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T18:57:39.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T18:57:39.870 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-14T18:57:39.871 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2023-07-14T18:57:39.897 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2023-07-14T18:57:39.925 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2023-07-14T18:57:39.970 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-14T18:57:44.965 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-07-14T18:57:44.965 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-14T18:57:44.988 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-14T18:57:44.991 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi133.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-14T19:11:58.523 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2023-07-14T19:11:58.525 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-07-14T19:11:58.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T19:12:03.596 DEBUG:teuthology.orchestra.run.smithi082:> true 2023-07-14T19:12:03.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-07-14T19:12:03.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T19:12:03.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T19:12:13.699 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-07-14T19:12:13.772 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-07-14T19:12:13.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T19:12:13.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T19:12:13.840 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-07-14T19:12:13.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-07-14T19:12:13.908 INFO:teuthology.run_tasks:Running task clock... 2023-07-14T19:12:13.921 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-14T19:12:13.922 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T19:12:13.922 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T19:12:13.925 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T19:12:13.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T19:12:13.928 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-14T19:12:13.928 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T19:12:13.959 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T19:12:13.963 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T19:12:13.974 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T19:12:13.979 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T19:12:13.981 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-14T19:12:13.996 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-14T19:12:14.047 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2023-07-14T19:12:14.054 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2023-07-14T19:12:14.059 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2023-07-14T19:12:14.063 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2023-07-14T19:12:14.067 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2023-07-14T19:12:14.075 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T19:12:14.075 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2023-07-14T19:12:14.081 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T19:12:14.089 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T19:12:14.090 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-14T19:12:14.096 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T19:12:14.104 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-14T19:12:14.168 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-07-14T19:12:14.171 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T19:12:14.171 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-07-14T19:12:14.171 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.171 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.172 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.172 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.173 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-07-14T19:12:14.177 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T19:12:14.177 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-07-14T19:12:14.177 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.177 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.177 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.177 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.178 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-07-14T19:12:14.181 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T19:12:14.181 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-07-14T19:12:14.182 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.182 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.182 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.182 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:12:14.183 INFO:teuthology.run_tasks:Running task pexec... 2023-07-14T19:12:14.192 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-14T19:12:14.193 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:12:14.193 DEBUG:teuthology.orchestra.run.smithi082:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T19:12:14.193 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T19:12:14.194 DEBUG:teuthology.orchestra.run.smithi133:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T19:12:14.194 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T19:12:14.194 DEBUG:teuthology.orchestra.run.smithi161:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-14T19:12:14.586 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:14.589 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:14.593 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:15.508 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:05:59 ago on Fri 14 Jul 2023 07:06:16 PM UTC. 2023-07-14T19:12:15.522 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:48 ago on Fri 14 Jul 2023 07:10:27 PM UTC. 2023-07-14T19:12:15.534 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:17 ago on Fri 14 Jul 2023 07:06:58 PM UTC. 2023-07-14T19:12:17.250 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T19:12:17.251 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T19:12:17.251 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2023-07-14T19:12:17.251 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T19:12:17.252 INFO:teuthology.orchestra.run.smithi161.stdout:Resetting modules: 2023-07-14T19:12:17.252 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools 2023-07-14T19:12:17.252 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:17.252 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T19:12:17.252 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-07-14T19:12:17.252 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:17.387 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T19:12:17.388 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-07-14T19:12:17.388 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2023-07-14T19:12:17.388 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-07-14T19:12:17.388 INFO:teuthology.orchestra.run.smithi133.stdout:Resetting modules: 2023-07-14T19:12:17.388 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools 2023-07-14T19:12:17.389 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:17.389 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-07-14T19:12:17.389 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-07-14T19:12:17.389 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:17.411 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-07-14T19:12:17.412 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-07-14T19:12:17.413 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2023-07-14T19:12:17.413 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-07-14T19:12:17.413 INFO:teuthology.orchestra.run.smithi082.stdout:Resetting modules: 2023-07-14T19:12:17.413 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools 2023-07-14T19:12:17.414 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:17.414 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-07-14T19:12:17.414 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-07-14T19:12:17.414 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:18.271 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:19 ago on Fri 14 Jul 2023 07:06:58 PM UTC. 2023-07-14T19:12:18.334 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:06:01 ago on Fri 14 Jul 2023 07:06:16 PM UTC. 2023-07-14T19:12:18.470 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:51 ago on Fri 14 Jul 2023 07:10:27 PM UTC. 2023-07-14T19:12:20.896 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T19:12:21.088 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T19:12:21.188 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-07-14T19:12:21.366 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:21.656 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:21.859 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:22.327 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:23 ago on Fri 14 Jul 2023 07:06:58 PM UTC. 2023-07-14T19:12:22.479 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:55 ago on Fri 14 Jul 2023 07:10:27 PM UTC. 2023-07-14T19:12:22.649 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:06:06 ago on Fri 14 Jul 2023 07:06:16 PM UTC. 2023-07-14T19:12:24.103 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T19:12:24.180 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-07-14T19:12:24.195 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T19:12:24.195 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-07-14T19:12:24.195 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout:Installing group/module packages: 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T19:12:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout:Installing module profiles: 2023-07-14T19:12:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools/common 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: container-tools rhel8 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================================================ 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout:Install 10 Packages 2023-07-14T19:12:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:24.199 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 18 M 2023-07-14T19:12:24.199 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 65 M 2023-07-14T19:12:24.199 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-07-14T19:12:24.269 INFO:teuthology.orchestra.run.smithi161.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-07-14T19:12:24.272 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout:Installing group/module packages: 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T19:12:24.273 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T19:12:24.274 INFO:teuthology.orchestra.run.smithi082.stdout:Installing module profiles: 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools/common 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools rhel8 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-07-14T19:12:24.275 INFO:teuthology.orchestra.run.smithi082.stdout:Install 10 Packages 2023-07-14T19:12:24.276 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:24.276 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 18 M 2023-07-14T19:12:24.276 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 65 M 2023-07-14T19:12:24.276 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-07-14T19:12:24.294 INFO:teuthology.orchestra.run.smithi161.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 259 kB/s | 25 kB 00:00 2023-07-14T19:12:24.319 INFO:teuthology.orchestra.run.smithi161.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 337 kB/s | 41 kB 00:00 2023-07-14T19:12:24.341 INFO:teuthology.orchestra.run.smithi082.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 619 kB/s | 41 kB 00:00 2023-07-14T19:12:24.344 INFO:teuthology.orchestra.run.smithi161.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-07-14T19:12:24.366 INFO:teuthology.orchestra.run.smithi082.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 273 kB/s | 25 kB 00:00 2023-07-14T19:12:24.378 INFO:teuthology.orchestra.run.smithi161.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-14T19:12:24.392 INFO:teuthology.orchestra.run.smithi082.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 987 kB/s | 49 kB 00:00 2023-07-14T19:12:24.404 INFO:teuthology.orchestra.run.smithi161.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-14T19:12:24.417 INFO:teuthology.orchestra.run.smithi082.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 663 kB/s | 94 kB 00:00 2023-07-14T19:12:24.429 INFO:teuthology.orchestra.run.smithi161.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-14T19:12:24.450 INFO:teuthology.orchestra.run.smithi082.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-14T19:12:24.476 INFO:teuthology.orchestra.run.smithi082.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-14T19:12:24.493 INFO:teuthology.orchestra.run.smithi082.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-07-14T19:12:24.512 INFO:teuthology.orchestra.run.smithi161.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-07-14T19:12:24.518 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T19:12:24.585 INFO:teuthology.orchestra.run.smithi082.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-07-14T19:12:24.617 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-07-14T19:12:24.617 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-07-14T19:12:24.617 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-07-14T19:12:24.617 INFO:teuthology.orchestra.run.smithi133.stdout:Installing group/module packages: 2023-07-14T19:12:24.617 INFO:teuthology.orchestra.run.smithi133.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-14T19:12:24.617 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-07-14T19:12:24.618 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout:Installing module profiles: 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools/common 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools rhel8 2023-07-14T19:12:24.619 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:24.620 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-07-14T19:12:24.621 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-07-14T19:12:24.621 INFO:teuthology.orchestra.run.smithi133.stdout:Install 10 Packages 2023-07-14T19:12:24.621 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:24.621 INFO:teuthology.orchestra.run.smithi161.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 32 MB/s | 7.0 MB 00:00 2023-07-14T19:12:24.622 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 18 M 2023-07-14T19:12:24.622 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 65 M 2023-07-14T19:12:24.622 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-07-14T19:12:24.701 INFO:teuthology.orchestra.run.smithi082.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-07-14T19:12:24.713 INFO:teuthology.orchestra.run.smithi161.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-07-14T19:12:24.713 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:12:24.714 INFO:teuthology.orchestra.run.smithi161.stdout:Total 35 MB/s | 18 MB 00:00 2023-07-14T19:12:24.802 INFO:teuthology.orchestra.run.smithi082.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-07-14T19:12:24.802 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:12:24.803 INFO:teuthology.orchestra.run.smithi082.stdout:Total 35 MB/s | 18 MB 00:00 2023-07-14T19:12:24.818 INFO:teuthology.orchestra.run.smithi133.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 125 kB/s | 25 kB 00:00 2023-07-14T19:12:24.843 INFO:teuthology.orchestra.run.smithi133.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 182 kB/s | 41 kB 00:00 2023-07-14T19:12:24.869 INFO:teuthology.orchestra.run.smithi133.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 377 kB/s | 94 kB 00:00 2023-07-14T19:12:24.894 INFO:teuthology.orchestra.run.smithi133.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-07-14T19:12:24.916 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-07-14T19:12:24.927 INFO:teuthology.orchestra.run.smithi133.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-14T19:12:24.935 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-07-14T19:12:24.935 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-07-14T19:12:24.953 INFO:teuthology.orchestra.run.smithi133.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-14T19:12:24.978 INFO:teuthology.orchestra.run.smithi133.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-14T19:12:25.002 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-07-14T19:12:25.021 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-07-14T19:12:25.022 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-07-14T19:12:25.070 INFO:teuthology.orchestra.run.smithi133.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-07-14T19:12:25.117 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-07-14T19:12:25.121 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-07-14T19:12:25.179 INFO:teuthology.orchestra.run.smithi133.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-07-14T19:12:25.204 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-07-14T19:12:25.208 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-07-14T19:12:25.262 INFO:teuthology.orchestra.run.smithi133.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-07-14T19:12:25.263 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:12:25.263 INFO:teuthology.orchestra.run.smithi133.stdout:Total 28 MB/s | 18 MB 00:00 2023-07-14T19:12:25.464 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-07-14T19:12:25.483 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-07-14T19:12:25.484 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-07-14T19:12:25.664 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-07-14T19:12:25.666 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-07-14T19:12:25.670 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-07-14T19:12:25.751 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-07-14T19:12:25.856 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T19:12:25.943 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T19:12:26.038 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T19:12:26.125 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T19:12:26.194 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-07-14T19:12:26.224 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T19:12:26.336 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T19:12:26.412 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-14T19:12:26.436 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T19:12:26.556 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T19:12:26.577 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T19:12:26.762 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-14T19:12:26.776 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T19:12:26.940 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-14T19:12:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T19:12:27.289 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-14T19:12:27.541 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T19:12:27.700 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T19:12:28.042 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-14T19:12:28.430 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T19:12:28.576 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T19:12:28.638 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T19:12:28.819 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T19:12:28.828 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T19:12:28.900 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-14T19:12:28.990 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T19:12:29.022 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T19:12:29.104 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-14T19:12:29.200 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T19:12:29.275 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-14T19:12:29.286 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T19:12:29.287 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T19:12:29.288 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T19:12:29.288 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T19:12:29.461 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T19:12:29.466 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T19:12:29.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T19:12:29.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T19:12:29.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T19:12:29.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T19:12:29.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T19:12:29.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T19:12:29.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T19:12:29.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T19:12:29.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T19:12:29.730 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-14T19:12:29.730 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-14T19:12:29.730 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-14T19:12:29.730 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-14T19:12:29.731 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-14T19:12:29.731 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-14T19:12:29.731 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-14T19:12:29.731 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-14T19:12:29.731 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-14T19:12:29.731 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-14T19:12:30.458 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T19:12:30.459 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:05:31 ago on Fri 14 Jul 2023 07:06:58 PM UTC. 2023-07-14T19:12:30.624 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T19:12:30.624 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:02:03 ago on Fri 14 Jul 2023 07:10:27 PM UTC. 2023-07-14T19:12:30.961 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-14T19:12:30.962 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:06:14 ago on Fri 14 Jul 2023 07:06:16 PM UTC. 2023-07-14T19:12:33.967 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-07-14T19:12:33.998 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-07-14T19:12:34.193 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:34.194 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-07-14T19:12:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T19:12:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T19:12:34.194 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:12:34.195 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T19:12:34.199 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:34.199 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T19:12:34.200 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.201 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.201 INFO:teuthology.orchestra.run.smithi082.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T19:12:34.201 INFO:teuthology.orchestra.run.smithi082.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T19:12:34.201 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:12:34.201 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-07-14T19:12:34.300 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-07-14T19:12:34.356 DEBUG:teuthology.parallel:result is None 2023-07-14T19:12:34.359 DEBUG:teuthology.parallel:result is None 2023-07-14T19:12:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:34.506 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-07-14T19:12:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T19:12:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:12:34.507 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T19:12:34.675 DEBUG:teuthology.parallel:result is None 2023-07-14T19:12:34.675 INFO:teuthology.run_tasks:Running task install... 2023-07-14T19:12:34.690 DEBUG:teuthology.task.install:project ceph 2023-07-14T19:12:34.691 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec'}} 2023-07-14T19:12:34.691 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec'} 2023-07-14T19:12:34.691 INFO:teuthology.task.install:Using flavor: default 2023-07-14T19:12:34.697 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-14T19:12:34.697 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-14T19:12:34.698 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'tag': None, 'wait_for_package': False} 2023-07-14T19:12:34.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:12:34.700 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'tag': None, 'wait_for_package': False} 2023-07-14T19:12:34.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:12:34.702 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'tag': None, 'wait_for_package': False} 2023-07-14T19:12:34.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:12:45.216 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/ 2023-07-14T19:12:45.217 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4940.g81c9387d 2023-07-14T19:12:45.322 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/ 2023-07-14T19:12:45.322 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4940.g81c9387d 2023-07-14T19:12:50.338 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/ 2023-07-14T19:12:50.338 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4940.g81c9387d 2023-07-14T19:12:55.600 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T19:12:55.601 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:12:55.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T19:12:55.634 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T19:12:55.634 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81c9387d2e7576da065747ec9c16d1c7d69016ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T19:12:55.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T19:12:55.719 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T19:12:55.719 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:12:55.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T19:12:55.753 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T19:12:55.754 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81c9387d2e7576da065747ec9c16d1c7d69016ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T19:12:55.793 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T19:12:55.820 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2023-07-14T19:12:55.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2023-07-14T19:12:55.825 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T19:12:55.841 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-14T19:12:55.841 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:12:55.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-14T19:12:55.874 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-14T19:12:55.874 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81c9387d2e7576da065747ec9c16d1c7d69016ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-14T19:12:55.907 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T19:12:55.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-14T19:12:55.974 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2023-07-14T19:12:55.976 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2023-07-14T19:12:56.023 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-14T19:12:56.091 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2023-07-14T19:12:56.093 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2023-07-14T19:12:56.191 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:56.354 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:56.443 INFO:teuthology.orchestra.run.smithi082.stdout:44 files removed 2023-07-14T19:12:56.465 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:56.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T19:12:56.627 INFO:teuthology.orchestra.run.smithi161.stdout:44 files removed 2023-07-14T19:12:56.674 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T19:12:56.744 INFO:teuthology.orchestra.run.smithi133.stdout:44 files removed 2023-07-14T19:12:56.790 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-14T19:12:56.818 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:57.007 INFO:teuthology.orchestra.run.smithi161.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:57.121 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-07-14T19:12:58.324 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 71 kB/s | 81 kB 00:01 2023-07-14T19:12:58.561 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 70 kB/s | 81 kB 00:01 2023-07-14T19:12:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 66 kB/s | 81 kB 00:01 2023-07-14T19:12:59.416 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 8.9 kB/s | 9.4 kB 00:01 2023-07-14T19:12:59.636 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 9.0 kB/s | 9.4 kB 00:01 2023-07-14T19:13:00.078 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 9.1 kB/s | 9.4 kB 00:01 2023-07-14T19:13:00.483 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 1.4 kB/s | 1.5 kB 00:01 2023-07-14T19:13:00.745 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 1.4 kB/s | 1.5 kB 00:01 2023-07-14T19:13:01.195 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 1.4 kB/s | 1.5 kB 00:01 2023-07-14T19:13:01.233 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 4.7 kB/s | 3.5 kB 00:00 2023-07-14T19:13:01.688 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 7.3 kB/s | 3.5 kB 00:00 2023-07-14T19:13:01.738 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 3.6 kB/s | 3.5 kB 00:00 2023-07-14T19:13:29.547 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 565 kB/s | 16 MB 00:28 2023-07-14T19:13:32.955 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 462 kB/s | 24 kB 00:00 2023-07-14T19:13:33.212 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 508 kB/s | 16 MB 00:31 2023-07-14T19:13:33.823 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-07-14T19:13:36.710 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 247 kB/s | 24 kB 00:00 2023-07-14T19:13:37.604 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-07-14T19:13:41.203 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 404 kB/s | 16 MB 00:39 2023-07-14T19:13:41.715 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-07-14T19:13:44.708 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 347 kB/s | 24 kB 00:00 2023-07-14T19:13:45.562 INFO:teuthology.orchestra.run.smithi161.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-07-14T19:13:45.585 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 66 MB/s | 53 MB 00:00 2023-07-14T19:13:52.787 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T19:13:52.787 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T19:13:52.801 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T19:13:52.802 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T19:13:52.803 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T19:13:52.804 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T19:13:52.806 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T19:13:52.809 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T19:13:52.811 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T19:13:52.944 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-07-14T19:13:52.956 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-07-14T19:13:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-07-14T19:13:52.957 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-07-14T19:13:52.957 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-07-14T19:13:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T19:13:52.957 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.4 k 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 5.4 M 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 850 k 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 167 k 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 1.6 M 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 129 k 2023-07-14T19:13:52.958 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 1.7 M 2023-07-14T19:13:52.959 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 7.4 M 2023-07-14T19:13:52.959 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 47 k 2023-07-14T19:13:52.959 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 14 M 2023-07-14T19:13:52.959 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 48 M 2023-07-14T19:13:52.959 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 258 k 2023-07-14T19:13:52.959 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 165 k 2023-07-14T19:13:52.960 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 260 k 2023-07-14T19:13:52.960 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 18 k 2023-07-14T19:13:52.960 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T19:13:52.960 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T19:13:52.961 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T19:13:52.962 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 31 k 2023-07-14T19:13:52.962 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 772 k 2023-07-14T19:13:52.962 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 139 k 2023-07-14T19:13:52.962 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T19:13:52.962 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T19:13:52.963 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T19:13:52.963 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T19:13:52.963 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T19:13:52.964 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T19:13:52.964 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 197 k 2023-07-14T19:13:52.964 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 371 k 2023-07-14T19:13:52.965 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 364 k 2023-07-14T19:13:52.965 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 107 k 2023-07-14T19:13:52.966 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 94 k 2023-07-14T19:13:52.966 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.5 M 2023-07-14T19:13:52.966 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 179 k 2023-07-14T19:13:52.966 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T19:13:52.967 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2023-07-14T19:13:52.967 INFO:teuthology.orchestra.run.smithi082.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T19:13:52.967 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.7 M 2023-07-14T19:13:52.967 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.6 M 2023-07-14T19:13:52.968 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-07-14T19:13:52.968 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T19:13:52.968 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 22 M 2023-07-14T19:13:52.969 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 23 k 2023-07-14T19:13:52.969 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 2.3 M 2023-07-14T19:13:52.969 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 241 k 2023-07-14T19:13:52.969 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 4.6 M 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 18 M 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 15 k 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 24 k 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T19:13:52.970 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 135 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T19:13:52.971 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 473 k 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.8 M 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T19:13:52.972 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T19:13:52.973 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T19:13:52.974 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T19:13:52.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T19:13:52.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T19:13:52.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T19:13:52.974 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T19:13:52.975 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T19:13:52.975 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 45 k 2023-07-14T19:13:52.975 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 122 k 2023-07-14T19:13:52.975 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T19:13:52.975 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T19:13:52.976 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T19:13:52.976 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T19:13:52.976 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T19:13:52.976 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T19:13:52.977 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T19:13:52.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T19:13:52.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T19:13:52.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T19:13:52.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T19:13:52.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T19:13:52.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T19:13:52.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T19:13:52.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T19:13:52.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T19:13:52.982 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql 10 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout:Install 128 Packages 2023-07-14T19:13:52.983 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 3 Packages 2023-07-14T19:13:52.984 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:13:52.984 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 206 M 2023-07-14T19:13:52.984 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-07-14T19:13:53.610 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-07-14T19:13:53.906 INFO:teuthology.orchestra.run.smithi082.stdout:(1/131): ceph-18.0.0-4940.g81c9387d.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-07-14T19:13:54.547 INFO:teuthology.orchestra.run.smithi082.stdout:(2/131): ceph-fuse-18.0.0-4940.g81c9387d.el8.x8 1.3 MB/s | 850 kB 00:00 2023-07-14T19:13:54.681 INFO:teuthology.orchestra.run.smithi082.stdout:(3/131): ceph-immutable-object-cache-18.0.0-494 1.2 MB/s | 167 kB 00:00 2023-07-14T19:13:55.373 INFO:teuthology.orchestra.run.smithi082.stdout:(4/131): ceph-mds-18.0.0-4940.g81c9387d.el8.x86 3.3 MB/s | 2.3 MB 00:00 2023-07-14T19:13:55.823 INFO:teuthology.orchestra.run.smithi082.stdout:(5/131): ceph-mgr-18.0.0-4940.g81c9387d.el8.x86 3.4 MB/s | 1.6 MB 00:00 2023-07-14T19:13:56.632 INFO:teuthology.orchestra.run.smithi082.stdout:(6/131): ceph-base-18.0.0-4940.g81c9387d.el8.x8 1.7 MB/s | 5.4 MB 00:03 2023-07-14T19:13:56.687 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T19:13:56.688 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T19:13:56.703 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T19:13:56.704 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T19:13:56.705 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T19:13:56.706 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T19:13:56.708 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T19:13:56.711 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T19:13:56.716 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T19:13:56.846 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-07-14T19:13:56.858 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T19:13:56.859 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-07-14T19:13:56.859 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T19:13:56.859 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-07-14T19:13:56.859 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T19:13:56.859 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T19:13:56.859 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.4 k 2023-07-14T19:13:56.860 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 5.4 M 2023-07-14T19:13:56.860 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 850 k 2023-07-14T19:13:56.860 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 167 k 2023-07-14T19:13:56.860 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 1.6 M 2023-07-14T19:13:56.860 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 129 k 2023-07-14T19:13:56.860 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 1.7 M 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 7.4 M 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 47 k 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 14 M 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 48 M 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 258 k 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 165 k 2023-07-14T19:13:56.861 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 260 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 18 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 31 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 772 k 2023-07-14T19:13:56.862 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 139 k 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T19:13:56.863 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 197 k 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 371 k 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 364 k 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 107 k 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 94 k 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.5 M 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 179 k 2023-07-14T19:13:56.864 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.7 M 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.6 M 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T19:13:56.865 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 22 M 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 23 k 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 2.3 M 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 241 k 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 4.6 M 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 18 M 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 15 k 2023-07-14T19:13:56.866 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 24 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 135 k 2023-07-14T19:13:56.867 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 473 k 2023-07-14T19:13:56.868 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.8 M 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T19:13:56.869 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T19:13:56.870 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 45 k 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 122 k 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T19:13:56.871 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T19:13:56.872 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T19:13:56.873 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T19:13:56.874 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T19:13:56.875 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T19:13:56.876 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T19:13:56.877 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout:Enabling module streams: 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql 10 2023-07-14T19:13:56.878 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:13:56.879 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-07-14T19:13:56.879 INFO:teuthology.orchestra.run.smithi161.stdout:=========================================================================================================================== 2023-07-14T19:13:56.879 INFO:teuthology.orchestra.run.smithi161.stdout:Install 128 Packages 2023-07-14T19:13:56.879 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 3 Packages 2023-07-14T19:13:56.879 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:13:56.880 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 206 M 2023-07-14T19:13:56.880 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-07-14T19:13:56.907 INFO:teuthology.orchestra.run.smithi082.stdout:(7/131): ceph-mon-18.0.0-4940.g81c9387d.el8.x86 4.3 MB/s | 4.6 MB 00:01 2023-07-14T19:13:58.187 INFO:teuthology.orchestra.run.smithi161.stdout:(1/131): ceph-18.0.0-4940.g81c9387d.el8.x86_64. 9.6 kB/s | 6.4 kB 00:00 2023-07-14T19:13:58.821 INFO:teuthology.orchestra.run.smithi161.stdout:(2/131): ceph-fuse-18.0.0-4940.g81c9387d.el8.x8 1.3 MB/s | 850 kB 00:00 2023-07-14T19:13:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:(3/131): ceph-immutable-object-cache-18.0.0-494 1.2 MB/s | 167 kB 00:00 2023-07-14T19:13:59.580 INFO:teuthology.orchestra.run.smithi161.stdout:(4/131): ceph-base-18.0.0-4940.g81c9387d.el8.x8 2.6 MB/s | 5.4 MB 00:02 2023-07-14T19:13:59.822 INFO:teuthology.orchestra.run.smithi161.stdout:(5/131): ceph-mds-18.0.0-4940.g81c9387d.el8.x86 2.6 MB/s | 2.3 MB 00:00 2023-07-14T19:13:59.905 INFO:teuthology.orchestra.run.smithi161.stdout:(6/131): ceph-mgr-18.0.0-4940.g81c9387d.el8.x86 4.8 MB/s | 1.6 MB 00:00 2023-07-14T19:14:01.530 INFO:teuthology.orchestra.run.smithi161.stdout:(7/131): ceph-mon-18.0.0-4940.g81c9387d.el8.x86 2.7 MB/s | 4.6 MB 00:01 2023-07-14T19:14:02.031 INFO:teuthology.orchestra.run.smithi161.stdout:(8/131): ceph-osd-18.0.0-4940.g81c9387d.el8.x86 8.4 MB/s | 18 MB 00:02 2023-07-14T19:14:02.123 INFO:teuthology.orchestra.run.smithi161.stdout:(9/131): ceph-selinux-18.0.0-4940.g81c9387d.el8 266 kB/s | 24 kB 00:00 2023-07-14T19:14:04.006 INFO:teuthology.orchestra.run.smithi082.stdout:(8/131): ceph-radosgw-18.0.0-4940.g81c9387d.el8 1.9 MB/s | 14 MB 00:07 2023-07-14T19:14:04.156 INFO:teuthology.orchestra.run.smithi082.stdout:(9/131): ceph-selinux-18.0.0-4940.g81c9387d.el8 162 kB/s | 24 kB 00:00 2023-07-14T19:14:04.625 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T19:14:04.626 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-14T19:14:04.641 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-14T19:14:04.642 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-14T19:14:04.643 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-14T19:14:04.644 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-14T19:14:04.646 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-14T19:14:04.649 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-14T19:14:04.650 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-14T19:14:04.782 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-07-14T19:14:04.794 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-07-14T19:14:04.795 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-07-14T19:14:04.795 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-07-14T19:14:04.795 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2023-07-14T19:14:04.795 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-14T19:14:04.795 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-14T19:14:04.795 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.4 k 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 5.4 M 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 850 k 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 167 k 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 1.6 M 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 129 k 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 1.7 M 2023-07-14T19:14:04.796 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 7.4 M 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 47 k 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 14 M 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 48 M 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 258 k 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 165 k 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 260 k 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 18 k 2023-07-14T19:14:04.797 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 31 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 772 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 139 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-14T19:14:04.798 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 197 k 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 371 k 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 364 k 2023-07-14T19:14:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 107 k 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 94 k 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.5 M 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 179 k 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.7 M 2023-07-14T19:14:04.800 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 3.6 M 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 22 M 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 23 k 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 2.3 M 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 241 k 2023-07-14T19:14:04.801 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 4.6 M 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 18 M 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4940.g81c9387d.el8 ceph-noarch 15 k 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 24 k 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-14T19:14:04.802 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 135 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-14T19:14:04.803 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 473 k 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 6.8 M 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-14T19:14:04.804 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-14T19:14:04.805 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 45 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:18.0.0-4940.g81c9387d.el8 ceph 122 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-14T19:14:04.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-14T19:14:04.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-14T19:14:04.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-14T19:14:04.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-14T19:14:04.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-14T19:14:04.811 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-14T19:14:04.812 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql 10 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout:Install 128 Packages 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 3 Packages 2023-07-14T19:14:04.813 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:14:04.814 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 206 M 2023-07-14T19:14:04.814 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-07-14T19:14:05.098 INFO:teuthology.orchestra.run.smithi161.stdout:(10/131): ceph-test-18.0.0-4940.g81c9387d.el8.x 16 MB/s | 48 MB 00:02 2023-07-14T19:14:05.256 INFO:teuthology.orchestra.run.smithi161.stdout:(11/131): cephfs-mirror-18.0.0-4940.g81c9387d.e 1.6 MB/s | 260 kB 00:00 2023-07-14T19:14:05.440 INFO:teuthology.orchestra.run.smithi161.stdout:(12/131): ceph-radosgw-18.0.0-4940.g81c9387d.el 3.5 MB/s | 14 MB 00:03 2023-07-14T19:14:05.465 INFO:teuthology.orchestra.run.smithi161.stdout:(13/131): libcephfs-devel-18.0.0-4940.g81c9387d 149 kB/s | 31 kB 00:00 2023-07-14T19:14:05.740 INFO:teuthology.orchestra.run.smithi161.stdout:(14/131): libcephfs2-18.0.0-4940.g81c9387d.el8. 2.5 MB/s | 772 kB 00:00 2023-07-14T19:14:05.827 INFO:teuthology.orchestra.run.smithi133.stdout:(1/131): ceph-18.0.0-4940.g81c9387d.el8.x86_64. 12 kB/s | 6.4 kB 00:00 2023-07-14T19:14:05.857 INFO:teuthology.orchestra.run.smithi161.stdout:(15/131): librados-devel-18.0.0-4940.g81c9387d. 1.2 MB/s | 139 kB 00:00 2023-07-14T19:14:05.882 INFO:teuthology.orchestra.run.smithi161.stdout:(16/131): libcephsqlite-18.0.0-4940.g81c9387d.e 325 kB/s | 135 kB 00:00 2023-07-14T19:14:06.083 INFO:teuthology.orchestra.run.smithi161.stdout:(17/131): libradosstriper1-18.0.0-4940.g81c9387 2.0 MB/s | 473 kB 00:00 2023-07-14T19:14:06.200 INFO:teuthology.orchestra.run.smithi161.stdout:(18/131): python3-ceph-argparse-18.0.0-4940.g81 384 kB/s | 45 kB 00:00 2023-07-14T19:14:06.808 INFO:teuthology.orchestra.run.smithi161.stdout:(19/131): python3-ceph-common-18.0.0-4940.g81c9 201 kB/s | 122 kB 00:00 2023-07-14T19:14:06.900 INFO:teuthology.orchestra.run.smithi161.stdout:(20/131): librgw2-18.0.0-4940.g81c9387d.el8.x86 6.7 MB/s | 6.8 MB 00:01 2023-07-14T19:14:06.926 INFO:teuthology.orchestra.run.smithi161.stdout:(21/131): python3-cephfs-18.0.0-4940.g81c9387d. 1.6 MB/s | 197 kB 00:00 2023-07-14T19:14:07.009 INFO:teuthology.orchestra.run.smithi161.stdout:(22/131): python3-rados-18.0.0-4940.g81c9387d.e 3.3 MB/s | 371 kB 00:00 2023-07-14T19:14:07.134 INFO:teuthology.orchestra.run.smithi161.stdout:(23/131): python3-rgw-18.0.0-4940.g81c9387d.el8 853 kB/s | 107 kB 00:00 2023-07-14T19:14:07.160 INFO:teuthology.orchestra.run.smithi161.stdout:(24/131): python3-rbd-18.0.0-4940.g81c9387d.el8 1.5 MB/s | 364 kB 00:00 2023-07-14T19:14:07.269 INFO:teuthology.orchestra.run.smithi161.stdout:(25/131): rbd-fuse-18.0.0-4940.g81c9387d.el8.x8 699 kB/s | 94 kB 00:00 2023-07-14T19:14:07.411 INFO:teuthology.orchestra.run.smithi161.stdout:(26/131): rbd-nbd-18.0.0-4940.g81c9387d.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-14T19:14:07.528 INFO:teuthology.orchestra.run.smithi161.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4940.g 198 kB/s | 23 kB 00:00 2023-07-14T19:14:07.653 INFO:teuthology.orchestra.run.smithi161.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4940.g81c9387 1.0 MB/s | 129 kB 00:00 2023-07-14T19:14:07.920 INFO:teuthology.orchestra.run.smithi161.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4940.g81c93 6.4 MB/s | 1.7 MB 00:00 2023-07-14T19:14:08.553 INFO:teuthology.orchestra.run.smithi161.stdout:(30/131): rbd-mirror-18.0.0-4940.g81c9387d.el8. 2.5 MB/s | 3.5 MB 00:01 2023-07-14T19:14:08.720 INFO:teuthology.orchestra.run.smithi161.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4940.g81 1.4 MB/s | 241 kB 00:00 2023-07-14T19:14:08.804 INFO:teuthology.orchestra.run.smithi161.stdout:(32/131): ceph-mgr-diskprediction-local-18.0.0- 8.4 MB/s | 7.4 MB 00:00 2023-07-14T19:14:08.863 INFO:teuthology.orchestra.run.smithi161.stdout:(33/131): ceph-mgr-rook-18.0.0-4940.g81c9387d.e 334 kB/s | 47 kB 00:00 2023-07-14T19:14:08.955 INFO:teuthology.orchestra.run.smithi161.stdout:(34/131): ceph-prometheus-alerts-18.0.0-4940.g8 97 kB/s | 15 kB 00:00 2023-07-14T19:14:08.997 INFO:teuthology.orchestra.run.smithi161.stdout:(35/131): ceph-volume-18.0.0-4940.g81c9387d.el8 1.9 MB/s | 258 kB 00:00 2023-07-14T19:14:09.001 INFO:teuthology.orchestra.run.smithi133.stdout:(2/131): ceph-fuse-18.0.0-4940.g81c9387d.el8.x8 268 kB/s | 850 kB 00:03 2023-07-14T19:14:09.072 INFO:teuthology.orchestra.run.smithi161.stdout:(36/131): cephadm-18.0.0-4940.g81c9387d.el8.noa 1.4 MB/s | 165 kB 00:00 2023-07-14T19:14:09.105 INFO:teuthology.orchestra.run.smithi161.stdout:(37/131): cephfs-top-18.0.0-4940.g81c9387d.el8. 166 kB/s | 18 kB 00:00 2023-07-14T19:14:09.335 INFO:teuthology.orchestra.run.smithi133.stdout:(3/131): ceph-immutable-object-cache-18.0.0-494 501 kB/s | 167 kB 00:00 2023-07-14T19:14:09.672 INFO:teuthology.orchestra.run.smithi161.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 104 kB/s | 59 kB 00:00 2023-07-14T19:14:09.948 INFO:teuthology.orchestra.run.smithi161.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 273 kB/s | 75 kB 00:00 2023-07-14T19:14:09.973 INFO:teuthology.orchestra.run.smithi161.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 340 kB/s | 306 kB 00:00 2023-07-14T19:14:10.132 INFO:teuthology.orchestra.run.smithi161.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 275 kB/s | 44 kB 00:00 2023-07-14T19:14:10.340 INFO:teuthology.orchestra.run.smithi161.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 889 kB/s | 185 kB 00:00 2023-07-14T19:14:10.424 INFO:teuthology.orchestra.run.smithi161.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2023-07-14T19:14:10.516 INFO:teuthology.orchestra.run.smithi161.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 188 kB/s | 33 kB 00:00 2023-07-14T19:14:10.808 INFO:teuthology.orchestra.run.smithi161.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 592 kB/s | 173 kB 00:00 2023-07-14T19:14:11.116 INFO:teuthology.orchestra.run.smithi161.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 24 kB/s | 16 kB 00:00 2023-07-14T19:14:11.191 INFO:teuthology.orchestra.run.smithi161.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.0 MB/s | 384 kB 00:00 2023-07-14T19:14:11.334 INFO:teuthology.orchestra.run.smithi161.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 78 kB/s | 11 kB 00:00 2023-07-14T19:14:11.367 INFO:teuthology.orchestra.run.smithi161.stdout:(49/131): python3-google-auth-1.1.1-10.el8.noar 324 kB/s | 81 kB 00:00 2023-07-14T19:14:11.459 INFO:teuthology.orchestra.run.smithi161.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-07-14T19:14:11.610 INFO:teuthology.orchestra.run.smithi161.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 324 kB/s | 49 kB 00:00 2023-07-14T19:14:11.743 INFO:teuthology.orchestra.run.smithi161.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 440 kB/s | 59 kB 00:00 2023-07-14T19:14:11.843 INFO:teuthology.orchestra.run.smithi161.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 603 kB/s | 60 kB 00:00 2023-07-14T19:14:12.202 INFO:teuthology.orchestra.run.smithi161.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 789 kB/s | 283 kB 00:00 2023-07-14T19:14:12.252 INFO:teuthology.orchestra.run.smithi161.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 1.3 MB/s | 1.2 MB 00:00 2023-07-14T19:14:12.436 INFO:teuthology.orchestra.run.smithi161.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 183 kB/s | 34 kB 00:00 2023-07-14T19:14:12.478 INFO:teuthology.orchestra.run.smithi161.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 57 kB/s | 16 kB 00:00 2023-07-14T19:14:12.670 INFO:teuthology.orchestra.run.smithi161.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 841 kB/s | 196 kB 00:00 2023-07-14T19:14:12.687 INFO:teuthology.orchestra.run.smithi161.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 295 kB/s | 61 kB 00:00 2023-07-14T19:14:12.812 INFO:teuthology.orchestra.run.smithi161.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 191 kB/s | 24 kB 00:00 2023-07-14T19:14:12.887 INFO:teuthology.orchestra.run.smithi161.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 86 kB/s | 19 kB 00:00 2023-07-14T19:14:12.929 INFO:teuthology.orchestra.run.smithi161.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-07-14T19:14:13.013 INFO:teuthology.orchestra.run.smithi161.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-07-14T19:14:13.238 INFO:teuthology.orchestra.run.smithi161.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 827 kB/s | 255 kB 00:00 2023-07-14T19:14:13.347 INFO:teuthology.orchestra.run.smithi161.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 753 kB/s | 251 kB 00:00 2023-07-14T19:14:13.397 INFO:teuthology.orchestra.run.smithi161.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 386 kB/s | 61 kB 00:00 2023-07-14T19:14:13.489 INFO:teuthology.orchestra.run.smithi161.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 602 kB/s | 85 kB 00:00 2023-07-14T19:14:13.514 INFO:teuthology.orchestra.run.smithi161.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-07-14T19:14:13.540 INFO:teuthology.orchestra.run.smithi161.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T19:14:13.873 INFO:teuthology.orchestra.run.smithi161.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 41 MB/s | 14 MB 00:00 2023-07-14T19:14:13.915 INFO:teuthology.orchestra.run.smithi161.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-14T19:14:13.957 INFO:teuthology.orchestra.run.smithi161.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-14T19:14:13.999 INFO:teuthology.orchestra.run.smithi161.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-14T19:14:14.041 INFO:teuthology.orchestra.run.smithi161.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-14T19:14:14.083 INFO:teuthology.orchestra.run.smithi161.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 929 kB/s | 39 kB 00:00 2023-07-14T19:14:14.392 INFO:teuthology.orchestra.run.smithi161.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 2.2 MB/s | 688 kB 00:00 2023-07-14T19:14:14.442 INFO:teuthology.orchestra.run.smithi161.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-07-14T19:14:14.484 INFO:teuthology.orchestra.run.smithi161.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-14T19:14:14.543 INFO:teuthology.orchestra.run.smithi161.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-07-14T19:14:14.593 INFO:teuthology.orchestra.run.smithi161.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-14T19:14:14.727 INFO:teuthology.orchestra.run.smithi161.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 36 MB/s | 4.8 MB 00:00 2023-07-14T19:14:14.760 INFO:teuthology.orchestra.run.smithi161.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 1.4 MB/s | 1.7 MB 00:01 2023-07-14T19:14:14.785 INFO:teuthology.orchestra.run.smithi161.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 454 kB/s | 27 kB 00:00 2023-07-14T19:14:14.844 INFO:teuthology.orchestra.run.smithi161.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 834 kB/s | 44 kB 00:00 2023-07-14T19:14:14.870 INFO:teuthology.orchestra.run.smithi161.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-07-14T19:14:14.920 INFO:teuthology.orchestra.run.smithi161.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-14T19:14:14.987 INFO:teuthology.orchestra.run.smithi161.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 26 MB/s | 3.7 MB 00:00 2023-07-14T19:14:15.012 INFO:teuthology.orchestra.run.smithi161.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-07-14T19:14:15.037 INFO:teuthology.orchestra.run.smithi161.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-14T19:14:15.054 INFO:teuthology.orchestra.run.smithi161.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-07-14T19:14:15.079 INFO:teuthology.orchestra.run.smithi161.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 480 kB/s | 20 kB 00:00 2023-07-14T19:14:15.105 INFO:teuthology.orchestra.run.smithi161.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-14T19:14:15.138 INFO:teuthology.orchestra.run.smithi161.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-14T19:14:15.188 INFO:teuthology.orchestra.run.smithi161.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-07-14T19:14:15.272 INFO:teuthology.orchestra.run.smithi161.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-07-14T19:14:15.322 INFO:teuthology.orchestra.run.smithi161.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-07-14T19:14:15.406 INFO:teuthology.orchestra.run.smithi161.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-07-14T19:14:15.431 INFO:teuthology.orchestra.run.smithi161.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-07-14T19:14:15.498 INFO:teuthology.orchestra.run.smithi161.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-07-14T19:14:15.590 INFO:teuthology.orchestra.run.smithi161.stdout:(100/131): postgresql-server-10.21-2.module+el8 28 MB/s | 5.1 MB 00:00 2023-07-14T19:14:15.640 INFO:teuthology.orchestra.run.smithi161.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-07-14T19:14:15.657 INFO:teuthology.orchestra.run.smithi161.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-07-14T19:14:15.699 INFO:teuthology.orchestra.run.smithi161.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-07-14T19:14:15.716 INFO:teuthology.orchestra.run.smithi161.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 725 kB/s | 43 kB 00:00 2023-07-14T19:14:15.750 INFO:teuthology.orchestra.run.smithi161.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-14T19:14:15.767 INFO:teuthology.orchestra.run.smithi161.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 960 kB/s | 48 kB 00:00 2023-07-14T19:14:15.809 INFO:teuthology.orchestra.run.smithi161.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-07-14T19:14:15.834 INFO:teuthology.orchestra.run.smithi161.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 634 kB/s | 43 kB 00:00 2023-07-14T19:14:15.885 INFO:teuthology.orchestra.run.smithi161.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.4 MB/s | 109 kB 00:00 2023-07-14T19:14:15.910 INFO:teuthology.orchestra.run.smithi161.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-07-14T19:14:15.943 INFO:teuthology.orchestra.run.smithi161.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-14T19:14:15.977 INFO:teuthology.orchestra.run.smithi161.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 280 kB/s | 19 kB 00:00 2023-07-14T19:14:16.002 INFO:teuthology.orchestra.run.smithi161.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.6 MB/s | 155 kB 00:00 2023-07-14T19:14:16.053 INFO:teuthology.orchestra.run.smithi161.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 7.3 MB/s | 559 kB 00:00 2023-07-14T19:14:16.078 INFO:teuthology.orchestra.run.smithi161.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 237 kB/s | 18 kB 00:00 2023-07-14T19:14:16.137 INFO:teuthology.orchestra.run.smithi161.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-07-14T19:14:16.162 INFO:teuthology.orchestra.run.smithi161.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-07-14T19:14:16.196 INFO:teuthology.orchestra.run.smithi161.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-07-14T19:14:16.246 INFO:teuthology.orchestra.run.smithi161.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 21 MB/s | 2.3 MB 00:00 2023-07-14T19:14:16.271 INFO:teuthology.orchestra.run.smithi161.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-07-14T19:14:16.297 INFO:teuthology.orchestra.run.smithi161.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-14T19:14:16.313 INFO:teuthology.orchestra.run.smithi161.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-07-14T19:14:16.339 INFO:teuthology.orchestra.run.smithi161.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-07-14T19:14:16.364 INFO:teuthology.orchestra.run.smithi161.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 766 kB/s | 39 kB 00:00 2023-07-14T19:14:16.398 INFO:teuthology.orchestra.run.smithi161.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-14T19:14:16.423 INFO:teuthology.orchestra.run.smithi161.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.8 MB/s | 171 kB 00:00 2023-07-14T19:14:16.448 INFO:teuthology.orchestra.run.smithi161.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-14T19:14:17.615 INFO:teuthology.orchestra.run.smithi161.stdout:(128/131): librbd1-18.0.0-4940.g81c9387d.el8.x8 3.1 MB/s | 3.6 MB 00:01 2023-07-14T19:14:17.711 INFO:teuthology.orchestra.run.smithi161.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 8.2 MB/s | 798 kB 00:00 2023-07-14T19:14:18.207 INFO:teuthology.orchestra.run.smithi161.stdout:(130/131): librados2-18.0.0-4940.g81c9387d.el8. 2.1 MB/s | 3.7 MB 00:01 2023-07-14T19:14:19.659 INFO:teuthology.orchestra.run.smithi133.stdout:(4/131): ceph-mds-18.0.0-4940.g81c9387d.el8.x86 228 kB/s | 2.3 MB 00:10 2023-07-14T19:14:21.332 INFO:teuthology.orchestra.run.smithi161.stdout:(131/131): ceph-common-18.0.0-4940.g81c9387d.el 953 kB/s | 22 MB 00:23 2023-07-14T19:14:21.336 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:14:21.336 INFO:teuthology.orchestra.run.smithi161.stdout:Total 8.4 MB/s | 206 MB 00:24 2023-07-14T19:14:21.492 INFO:teuthology.orchestra.run.smithi133.stdout:(5/131): ceph-base-18.0.0-4940.g81c9387d.el8.x8 340 kB/s | 5.4 MB 00:16 2023-07-14T19:14:22.177 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-07-14T19:14:22.245 INFO:teuthology.orchestra.run.smithi082.stdout:(10/131): ceph-osd-18.0.0-4940.g81c9387d.el8.x8 710 kB/s | 18 MB 00:25 2023-07-14T19:14:22.297 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-07-14T19:14:22.298 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-07-14T19:14:22.537 INFO:teuthology.orchestra.run.smithi082.stdout:(11/131): cephfs-mirror-18.0.0-4940.g81c9387d.e 893 kB/s | 260 kB 00:00 2023-07-14T19:14:22.679 INFO:teuthology.orchestra.run.smithi082.stdout:(12/131): libcephfs-devel-18.0.0-4940.g81c9387d 219 kB/s | 31 kB 00:00 2023-07-14T19:14:23.109 INFO:teuthology.orchestra.run.smithi133.stdout:(6/131): ceph-mgr-18.0.0-4940.g81c9387d.el8.x86 460 kB/s | 1.6 MB 00:03 2023-07-14T19:14:23.411 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-07-14T19:14:23.416 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-07-14T19:14:23.754 INFO:teuthology.orchestra.run.smithi082.stdout:(13/131): libcephfs2-18.0.0-4940.g81c9387d.el8. 719 kB/s | 772 kB 00:01 2023-07-14T19:14:23.987 INFO:teuthology.orchestra.run.smithi082.stdout:(14/131): libcephsqlite-18.0.0-4940.g81c9387d.e 581 kB/s | 135 kB 00:00 2023-07-14T19:14:24.221 INFO:teuthology.orchestra.run.smithi082.stdout:(15/131): librados-devel-18.0.0-4940.g81c9387d. 596 kB/s | 139 kB 00:00 2023-07-14T19:14:25.171 INFO:teuthology.orchestra.run.smithi082.stdout:(16/131): libradosstriper1-18.0.0-4940.g81c9387 498 kB/s | 473 kB 00:00 2023-07-14T19:14:28.421 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-07-14T19:14:28.620 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T19:14:28.800 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T19:14:29.029 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T19:14:29.206 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T19:14:29.322 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T19:14:29.556 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T19:14:29.960 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T19:14:30.147 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:14:30.545 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:14:30.731 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:14:30.840 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:14:31.006 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:14:31.088 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:14:31.286 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T19:14:31.509 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T19:14:31.708 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T19:14:32.076 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T19:14:32.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T19:14:32.463 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T19:14:32.577 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T19:14:32.706 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T19:14:32.779 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T19:14:32.914 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T19:14:33.066 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T19:14:33.222 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T19:14:33.824 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T19:14:33.980 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T19:14:34.565 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T19:14:34.746 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T19:14:35.435 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T19:14:35.695 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T19:14:35.922 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T19:14:36.150 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T19:14:36.336 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T19:14:36.543 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T19:14:36.773 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T19:14:36.916 INFO:teuthology.orchestra.run.smithi133.stdout:(7/131): ceph-mon-18.0.0-4940.g81c9387d.el8.x86 307 kB/s | 4.6 MB 00:15 2023-07-14T19:14:36.953 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 26/134 2023-07-14T19:14:37.074 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 27/134 2023-07-14T19:14:37.989 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 28/134 2023-07-14T19:14:38.094 INFO:teuthology.orchestra.run.smithi082.stdout:(17/131): ceph-common-18.0.0-4940.g81c9387d.el8 510 kB/s | 22 MB 00:44 2023-07-14T19:14:38.252 INFO:teuthology.orchestra.run.smithi082.stdout:(18/131): python3-ceph-argparse-18.0.0-4940.g81 283 kB/s | 45 kB 00:00 2023-07-14T19:14:38.311 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T19:14:38.503 INFO:teuthology.orchestra.run.smithi082.stdout:(19/131): python3-ceph-common-18.0.0-4940.g81c9 490 kB/s | 122 kB 00:00 2023-07-14T19:14:38.606 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T19:14:38.871 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T19:14:38.903 INFO:teuthology.orchestra.run.smithi082.stdout:(20/131): python3-cephfs-18.0.0-4940.g81c9387d. 492 kB/s | 197 kB 00:00 2023-07-14T19:14:39.046 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T19:14:39.745 INFO:teuthology.orchestra.run.smithi082.stdout:(21/131): python3-rados-18.0.0-4940.g81c9387d.e 441 kB/s | 371 kB 00:00 2023-07-14T19:14:40.728 INFO:teuthology.orchestra.run.smithi082.stdout:(22/131): python3-rbd-18.0.0-4940.g81c9387d.el8 370 kB/s | 364 kB 00:00 2023-07-14T19:14:40.810 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T19:14:40.978 INFO:teuthology.orchestra.run.smithi082.stdout:(23/131): python3-rgw-18.0.0-4940.g81c9387d.el8 426 kB/s | 107 kB 00:00 2023-07-14T19:14:41.081 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T19:14:41.212 INFO:teuthology.orchestra.run.smithi082.stdout:(24/131): rbd-fuse-18.0.0-4940.g81c9387d.el8.x8 401 kB/s | 94 kB 00:00 2023-07-14T19:14:41.266 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T19:14:41.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T19:14:41.635 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 36/134 2023-07-14T19:14:41.841 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 37/134 2023-07-14T19:14:42.002 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T19:14:42.188 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T19:14:42.356 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T19:14:42.499 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T19:14:42.726 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T19:14:42.924 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T19:14:43.291 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T19:14:43.582 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T19:14:43.780 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T19:14:43.999 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T19:14:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T19:14:44.546 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T19:14:44.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T19:14:44.754 INFO:teuthology.orchestra.run.smithi082.stdout:(25/131): librgw2-18.0.0-4940.g81c9387d.el8.x86 355 kB/s | 6.8 MB 00:19 2023-07-14T19:14:44.938 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T19:14:45.095 INFO:teuthology.orchestra.run.smithi082.stdout:(26/131): rbd-nbd-18.0.0-4940.g81c9387d.el8.x86 524 kB/s | 179 kB 00:00 2023-07-14T19:14:45.150 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T19:14:45.179 INFO:teuthology.orchestra.run.smithi082.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4940.g 277 kB/s | 23 kB 00:00 2023-07-14T19:14:45.291 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T19:14:45.457 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T19:14:45.479 INFO:teuthology.orchestra.run.smithi082.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4940.g81c9387 431 kB/s | 129 kB 00:00 2023-07-14T19:14:45.736 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T19:14:45.967 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T19:14:46.113 INFO:teuthology.orchestra.run.smithi082.stdout:(29/131): rbd-mirror-18.0.0-4940.g81c9387d.el8. 738 kB/s | 3.5 MB 00:04 2023-07-14T19:14:46.200 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T19:14:46.418 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T19:14:46.610 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T19:14:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T19:14:46.895 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T19:14:46.965 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T19:14:47.197 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T19:14:47.398 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T19:14:47.545 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T19:14:47.627 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T19:14:47.859 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T19:14:48.030 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T19:14:48.196 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T19:14:48.375 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T19:14:48.628 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T19:14:48.843 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T19:14:48.991 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T19:14:49.131 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T19:14:49.313 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T19:14:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T19:14:49.657 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T19:14:49.839 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T19:14:49.959 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T19:14:50.175 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T19:14:50.407 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T19:14:50.567 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T19:14:50.801 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T19:14:50.994 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T19:14:51.160 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T19:14:51.413 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T19:14:51.487 INFO:teuthology.orchestra.run.smithi082.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4940.g81c93 292 kB/s | 1.7 MB 00:06 2023-07-14T19:14:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T19:14:52.654 INFO:teuthology.orchestra.run.smithi082.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4940.g81 206 kB/s | 241 kB 00:01 2023-07-14T19:14:52.747 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T19:14:52.854 INFO:teuthology.orchestra.run.smithi082.stdout:(32/131): ceph-mgr-rook-18.0.0-4940.g81c9387d.e 237 kB/s | 47 kB 00:00 2023-07-14T19:14:52.921 INFO:teuthology.orchestra.run.smithi082.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4940.g8 218 kB/s | 15 kB 00:00 2023-07-14T19:14:52.994 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T19:14:53.296 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T19:14:53.479 INFO:teuthology.orchestra.run.smithi082.stdout:(34/131): ceph-volume-18.0.0-4940.g81c9387d.el8 463 kB/s | 258 kB 00:00 2023-07-14T19:14:53.570 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T19:14:53.746 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T19:14:53.813 INFO:teuthology.orchestra.run.smithi082.stdout:(35/131): cephadm-18.0.0-4940.g81c9387d.el8.noa 496 kB/s | 165 kB 00:00 2023-07-14T19:14:53.905 INFO:teuthology.orchestra.run.smithi082.stdout:(36/131): cephfs-top-18.0.0-4940.g81c9387d.el8. 196 kB/s | 18 kB 00:00 2023-07-14T19:14:54.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T19:14:54.719 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T19:14:54.780 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T19:14:54.805 INFO:teuthology.orchestra.run.smithi082.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 340 kB/s | 306 kB 00:00 2023-07-14T19:14:54.927 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 93/134 2023-07-14T19:14:55.080 INFO:teuthology.orchestra.run.smithi082.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 214 kB/s | 59 kB 00:00 2023-07-14T19:14:55.231 INFO:teuthology.orchestra.run.smithi082.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 501 kB/s | 75 kB 00:00 2023-07-14T19:14:56.222 INFO:teuthology.orchestra.run.smithi082.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 480 kB/s | 476 kB 00:00 2023-07-14T19:14:56.356 INFO:teuthology.orchestra.run.smithi082.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2023-07-14T19:14:56.748 INFO:teuthology.orchestra.run.smithi082.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 473 kB/s | 185 kB 00:00 2023-07-14T19:14:56.890 INFO:teuthology.orchestra.run.smithi082.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 233 kB/s | 33 kB 00:00 2023-07-14T19:14:57.032 INFO:teuthology.orchestra.run.smithi082.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 115 kB/s | 16 kB 00:00 2023-07-14T19:14:57.407 INFO:teuthology.orchestra.run.smithi082.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 460 kB/s | 173 kB 00:00 2023-07-14T19:14:58.583 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:14:58.657 INFO:teuthology.orchestra.run.smithi082.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 308 kB/s | 384 kB 00:01 2023-07-14T19:14:58.793 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:14:59.041 INFO:teuthology.orchestra.run.smithi082.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 212 kB/s | 81 kB 00:00 2023-07-14T19:14:59.166 INFO:teuthology.orchestra.run.smithi082.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-07-14T19:14:59.325 INFO:teuthology.orchestra.run.smithi082.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 111 kB/s | 18 kB 00:00 2023-07-14T19:14:59.374 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:14:59.614 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T19:14:59.690 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T19:14:59.826 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T19:15:02.524 INFO:teuthology.orchestra.run.smithi082.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 373 kB/s | 1.2 MB 00:03 2023-07-14T19:15:02.741 INFO:teuthology.orchestra.run.smithi082.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 225 kB/s | 49 kB 00:00 2023-07-14T19:15:02.992 INFO:teuthology.orchestra.run.smithi082.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 234 kB/s | 59 kB 00:00 2023-07-14T19:15:03.233 INFO:teuthology.orchestra.run.smithi082.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 249 kB/s | 60 kB 00:00 2023-07-14T19:15:04.092 INFO:teuthology.orchestra.run.smithi082.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 329 kB/s | 283 kB 00:00 2023-07-14T19:15:04.226 INFO:teuthology.orchestra.run.smithi082.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2023-07-14T19:15:04.343 INFO:teuthology.orchestra.run.smithi082.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 288 kB/s | 34 kB 00:00 2023-07-14T19:15:04.826 INFO:teuthology.orchestra.run.smithi082.stdout:(57/131): ceph-mgr-diskprediction-local-18.0.0- 405 kB/s | 7.4 MB 00:18 2023-07-14T19:15:05.093 INFO:teuthology.orchestra.run.smithi082.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 262 kB/s | 196 kB 00:00 2023-07-14T19:15:05.252 INFO:teuthology.orchestra.run.smithi082.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 119 kB/s | 19 kB 00:00 2023-07-14T19:15:05.419 INFO:teuthology.orchestra.run.smithi082.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 143 kB/s | 24 kB 00:00 2023-07-14T19:15:05.561 INFO:teuthology.orchestra.run.smithi082.stdout:(61/131): python3-rsa-4.8-1.el8.noarch.rpm 84 kB/s | 61 kB 00:00 2023-07-14T19:15:05.594 INFO:teuthology.orchestra.run.smithi082.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 167 kB/s | 29 kB 00:00 2023-07-14T19:15:05.744 INFO:teuthology.orchestra.run.smithi082.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 150 kB/s | 27 kB 00:00 2023-07-14T19:15:06.478 INFO:teuthology.orchestra.run.smithi082.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 342 kB/s | 251 kB 00:00 2023-07-14T19:15:06.678 INFO:teuthology.orchestra.run.smithi082.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 306 kB/s | 61 kB 00:00 2023-07-14T19:15:06.787 INFO:teuthology.orchestra.run.smithi082.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 214 kB/s | 255 kB 00:01 2023-07-14T19:15:06.962 INFO:teuthology.orchestra.run.smithi082.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 131 kB/s | 23 kB 00:00 2023-07-14T19:15:07.012 INFO:teuthology.orchestra.run.smithi082.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 256 kB/s | 85 kB 00:00 2023-07-14T19:15:07.037 INFO:teuthology.orchestra.run.smithi082.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-14T19:15:07.266 INFO:teuthology.orchestra.run.smithi133.stdout:(8/131): ceph-radosgw-18.0.0-4940.g81c9387d.el8 457 kB/s | 14 MB 00:30 2023-07-14T19:15:07.338 INFO:teuthology.orchestra.run.smithi082.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 46 MB/s | 14 MB 00:00 2023-07-14T19:15:07.370 INFO:teuthology.orchestra.run.smithi133.stdout:(9/131): ceph-selinux-18.0.0-4940.g81c9387d.el8 234 kB/s | 24 kB 00:00 2023-07-14T19:15:07.380 INFO:teuthology.orchestra.run.smithi082.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-14T19:15:07.422 INFO:teuthology.orchestra.run.smithi082.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-14T19:15:07.464 INFO:teuthology.orchestra.run.smithi082.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-14T19:15:07.506 INFO:teuthology.orchestra.run.smithi082.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-14T19:15:07.548 INFO:teuthology.orchestra.run.smithi082.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 930 kB/s | 39 kB 00:00 2023-07-14T19:15:07.598 INFO:teuthology.orchestra.run.smithi082.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-14T19:15:07.648 INFO:teuthology.orchestra.run.smithi082.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-07-14T19:15:07.682 INFO:teuthology.orchestra.run.smithi082.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-07-14T19:15:07.723 INFO:teuthology.orchestra.run.smithi082.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-14T19:15:07.799 INFO:teuthology.orchestra.run.smithi082.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 9.2 MB/s | 713 kB 00:00 2023-07-14T19:15:07.933 INFO:teuthology.orchestra.run.smithi082.stdout:(81/131): python3-babel-2.5.1-7.el8.noarch.rpm 36 MB/s | 4.8 MB 00:00 2023-07-14T19:15:07.975 INFO:teuthology.orchestra.run.smithi082.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 638 kB/s | 27 kB 00:00 2023-07-14T19:15:08.017 INFO:teuthology.orchestra.run.smithi082.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-14T19:15:08.059 INFO:teuthology.orchestra.run.smithi082.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-07-14T19:15:08.226 INFO:teuthology.orchestra.run.smithi082.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-07-14T19:15:08.276 INFO:teuthology.orchestra.run.smithi082.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-14T19:15:08.318 INFO:teuthology.orchestra.run.smithi082.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-14T19:15:08.368 INFO:teuthology.orchestra.run.smithi082.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-14T19:15:08.410 INFO:teuthology.orchestra.run.smithi082.stdout:(89/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-07-14T19:15:08.444 INFO:teuthology.orchestra.run.smithi082.stdout:(90/131): python3-jsonpointer-1.10-11.el8.noarc 599 kB/s | 20 kB 00:00 2023-07-14T19:15:08.569 INFO:teuthology.orchestra.run.smithi082.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 829 kB/s | 103 kB 00:00 2023-07-14T19:15:08.761 INFO:teuthology.orchestra.run.smithi082.stdout:(92/131): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2023-07-14T19:15:09.003 INFO:teuthology.orchestra.run.smithi082.stdout:(93/131): openblas-threads-0.3.15-3.el8.x86_64. 20 MB/s | 4.9 MB 00:00 2023-07-14T19:15:09.179 INFO:teuthology.orchestra.run.smithi133.stdout:(10/131): ceph-common-18.0.0-4940.g81c9387d.el8 355 kB/s | 22 MB 01:03 2023-07-14T19:15:09.203 INFO:teuthology.orchestra.run.smithi082.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 989 kB/s | 198 kB 00:00 2023-07-14T19:15:09.420 INFO:teuthology.orchestra.run.smithi082.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-07-14T19:15:09.520 INFO:teuthology.orchestra.run.smithi082.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2023-07-14T19:15:09.654 INFO:teuthology.orchestra.run.smithi082.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-07-14T19:15:09.838 INFO:teuthology.orchestra.run.smithi082.stdout:(98/131): postgresql-server-10.21-2.module+el8. 28 MB/s | 5.1 MB 00:00 2023-07-14T19:15:09.980 INFO:teuthology.orchestra.run.smithi082.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 5.6 MB/s | 808 kB 00:00 2023-07-14T19:15:10.124 INFO:teuthology.orchestra.run.smithi082.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-07-14T19:15:10.330 INFO:teuthology.orchestra.run.smithi082.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 225 kB/s | 47 kB 00:00 2023-07-14T19:15:10.697 INFO:teuthology.orchestra.run.smithi082.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 545 kB/s | 200 kB 00:00 2023-07-14T19:15:10.947 INFO:teuthology.orchestra.run.smithi082.stdout:(103/131): python3-jwt-1.6.1-2.el8.noarch.rpm 170 kB/s | 43 kB 00:00 2023-07-14T19:15:11.078 INFO:teuthology.orchestra.run.smithi133.stdout:(11/131): cephfs-mirror-18.0.0-4940.g81c9387d.e 137 kB/s | 260 kB 00:01 2023-07-14T19:15:11.098 INFO:teuthology.orchestra.run.smithi082.stdout:(104/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-07-14T19:15:11.298 INFO:teuthology.orchestra.run.smithi082.stdout:(105/131): keyutils-libs-devel-1.5.10-9.el8.x86 241 kB/s | 48 kB 00:00 2023-07-14T19:15:11.312 INFO:teuthology.orchestra.run.smithi133.stdout:(12/131): libcephfs-devel-18.0.0-4940.g81c9387d 133 kB/s | 31 kB 00:00 2023-07-14T19:15:11.557 INFO:teuthology.orchestra.run.smithi082.stdout:(106/131): libsepol-devel-2.9-3.el8.x86_64.rpm 336 kB/s | 87 kB 00:00 2023-07-14T19:15:11.674 INFO:teuthology.orchestra.run.smithi082.stdout:(107/131): python3-requests-oauthlib-1.0.0-1.el 366 kB/s | 43 kB 00:00 2023-07-14T19:15:11.766 INFO:teuthology.orchestra.run.smithi082.stdout:(108/131): python3-pycparser-2.14-14.el8.noarch 1.2 MB/s | 109 kB 00:00 2023-07-14T19:15:11.841 INFO:teuthology.orchestra.run.smithi082.stdout:(109/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-07-14T19:15:11.908 INFO:teuthology.orchestra.run.smithi082.stdout:(110/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-07-14T19:15:11.983 INFO:teuthology.orchestra.run.smithi082.stdout:(111/131): libaio-devel-0.3.112-1.el8.x86_64.rp 250 kB/s | 19 kB 00:00 2023-07-14T19:15:12.042 INFO:teuthology.orchestra.run.smithi082.stdout:(112/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.6 MB/s | 155 kB 00:00 2023-07-14T19:15:12.109 INFO:teuthology.orchestra.run.smithi082.stdout:(113/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-07-14T19:15:12.168 INFO:teuthology.orchestra.run.smithi082.stdout:(114/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-07-14T19:15:12.243 INFO:teuthology.orchestra.run.smithi082.stdout:(115/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.9 MB/s | 605 kB 00:00 2023-07-14T19:15:12.318 INFO:teuthology.orchestra.run.smithi082.stdout:(116/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.1 MB/s | 238 kB 00:00 2023-07-14T19:15:12.435 INFO:teuthology.orchestra.run.smithi082.stdout:(117/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 20 MB/s | 2.3 MB 00:00 2023-07-14T19:15:12.494 INFO:teuthology.orchestra.run.smithi082.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-07-14T19:15:12.552 INFO:teuthology.orchestra.run.smithi082.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-07-14T19:15:12.604 INFO:teuthology.orchestra.run.smithi082.stdout:(120/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-14T19:15:12.653 INFO:teuthology.orchestra.run.smithi082.stdout:(121/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-14T19:15:12.712 INFO:teuthology.orchestra.run.smithi082.stdout:(122/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 992 kB/s | 58 kB 00:00 2023-07-14T19:15:12.762 INFO:teuthology.orchestra.run.smithi082.stdout:(123/131): libcom_err-devel-1.45.6-4.el8.x86_64 771 kB/s | 39 kB 00:00 2023-07-14T19:15:12.846 INFO:teuthology.orchestra.run.smithi082.stdout:(124/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 7.5 MB/s | 644 kB 00:00 2023-07-14T19:15:12.905 INFO:teuthology.orchestra.run.smithi082.stdout:(125/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-14T19:15:12.963 INFO:teuthology.orchestra.run.smithi082.stdout:(126/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-07-14T19:15:15.045 INFO:teuthology.orchestra.run.smithi133.stdout:(13/131): libcephfs2-18.0.0-4940.g81c9387d.el8. 207 kB/s | 772 kB 00:03 2023-07-14T19:15:15.463 INFO:teuthology.orchestra.run.smithi133.stdout:(14/131): libcephsqlite-18.0.0-4940.g81c9387d.e 325 kB/s | 135 kB 00:00 2023-07-14T19:15:15.554 INFO:teuthology.orchestra.run.smithi082.stdout:(127/131): thrift-0.13.0-2.el8.x86_64.rpm 207 kB/s | 1.7 MB 00:08 2023-07-14T19:15:15.796 INFO:teuthology.orchestra.run.smithi133.stdout:(15/131): librados-devel-18.0.0-4940.g81c9387d. 418 kB/s | 139 kB 00:00 2023-07-14T19:15:16.029 INFO:teuthology.orchestra.run.smithi133.stdout:(16/131): ceph-osd-18.0.0-4940.g81c9387d.el8.x8 344 kB/s | 18 MB 00:52 2023-07-14T19:15:17.113 INFO:teuthology.orchestra.run.smithi133.stdout:(17/131): libradosstriper1-18.0.0-4940.g81c9387 359 kB/s | 473 kB 00:01 2023-07-14T19:15:17.247 INFO:teuthology.orchestra.run.smithi133.stdout:(18/131): python3-ceph-argparse-18.0.0-4940.g81 335 kB/s | 45 kB 00:00 2023-07-14T19:15:17.563 INFO:teuthology.orchestra.run.smithi133.stdout:(19/131): python3-ceph-common-18.0.0-4940.g81c9 387 kB/s | 122 kB 00:00 2023-07-14T19:15:18.072 INFO:teuthology.orchestra.run.smithi133.stdout:(20/131): python3-cephfs-18.0.0-4940.g81c9387d. 387 kB/s | 197 kB 00:00 2023-07-14T19:15:18.412 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T19:15:18.412 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2023-07-14T19:15:18.413 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2023-07-14T19:15:18.413 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-07-14T19:15:18.413 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2023-07-14T19:15:18.413 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2023-07-14T19:15:18.414 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2023-07-14T19:15:18.414 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2023-07-14T19:15:18.414 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2023-07-14T19:15:18.414 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-07-14T19:15:18.415 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:15:18.732 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T19:15:19.072 INFO:teuthology.orchestra.run.smithi133.stdout:(21/131): python3-rados-18.0.0-4940.g81c9387d.e 371 kB/s | 371 kB 00:00 2023-07-14T19:15:20.316 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T19:15:20.381 INFO:teuthology.orchestra.run.smithi133.stdout:(22/131): python3-rbd-18.0.0-4940.g81c9387d.el8 278 kB/s | 364 kB 00:01 2023-07-14T19:15:20.485 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T19:15:20.705 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T19:15:20.839 INFO:teuthology.orchestra.run.smithi133.stdout:(23/131): python3-rgw-18.0.0-4940.g81c9387d.el8 233 kB/s | 107 kB 00:00 2023-07-14T19:15:20.873 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T19:15:21.231 INFO:teuthology.orchestra.run.smithi133.stdout:(24/131): rbd-fuse-18.0.0-4940.g81c9387d.el8.x8 239 kB/s | 94 kB 00:00 2023-07-14T19:15:21.340 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T19:15:21.515 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T19:15:21.535 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T19:15:23.191 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T19:15:23.287 INFO:teuthology.orchestra.run.smithi082.stdout:(128/131): librados2-18.0.0-4940.g81c9387d.el8. 365 kB/s | 3.7 MB 00:10 2023-07-14T19:15:23.337 INFO:teuthology.orchestra.run.smithi082.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-14T19:15:23.346 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T19:15:23.470 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 102/134 2023-07-14T19:15:23.662 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 103/134 2023-07-14T19:15:23.737 INFO:teuthology.orchestra.run.smithi082.stdout:(130/131): librbd1-18.0.0-4940.g81c9387d.el8.x8 454 kB/s | 3.6 MB 00:08 2023-07-14T19:15:23.855 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T19:15:24.074 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T19:15:24.351 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T19:15:25.314 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T19:15:25.489 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T19:15:25.580 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T19:15:25.913 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 107/134 2023-07-14T19:15:26.104 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T19:15:26.140 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T19:15:26.272 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T19:15:26.283 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T19:15:26.747 INFO:teuthology.orchestra.run.smithi133.stdout:(25/131): rbd-mirror-18.0.0-4940.g81c9387d.el8. 655 kB/s | 3.5 MB 00:05 2023-07-14T19:15:27.056 INFO:teuthology.orchestra.run.smithi133.stdout:(26/131): rbd-nbd-18.0.0-4940.g81c9387d.el8.x86 581 kB/s | 179 kB 00:00 2023-07-14T19:15:27.139 INFO:teuthology.orchestra.run.smithi133.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4940.g 277 kB/s | 23 kB 00:00 2023-07-14T19:15:27.381 INFO:teuthology.orchestra.run.smithi133.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4940.g81c9387 535 kB/s | 129 kB 00:00 2023-07-14T19:15:27.568 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 110/134 2023-07-14T19:15:27.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T19:15:27.780 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T19:15:27.947 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T19:15:27.989 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T19:15:28.147 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T19:15:28.504 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T19:15:28.672 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T19:15:30.856 INFO:teuthology.orchestra.run.smithi133.stdout:(29/131): librgw2-18.0.0-4940.g81c9387d.el8.x86 468 kB/s | 6.8 MB 00:14 2023-07-14T19:15:31.098 INFO:teuthology.orchestra.run.smithi133.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4940.g81c93 473 kB/s | 1.7 MB 00:03 2023-07-14T19:15:31.865 INFO:teuthology.orchestra.run.smithi133.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4940.g81 314 kB/s | 241 kB 00:00 2023-07-14T19:15:32.123 INFO:teuthology.orchestra.run.smithi133.stdout:(32/131): ceph-mgr-rook-18.0.0-4940.g81c9387d.e 183 kB/s | 47 kB 00:00 2023-07-14T19:15:32.240 INFO:teuthology.orchestra.run.smithi133.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4940.g8 124 kB/s | 15 kB 00:00 2023-07-14T19:15:32.827 INFO:teuthology.orchestra.run.smithi082.stdout:(131/131): ceph-test-18.0.0-4940.g81c9387d.el8. 554 kB/s | 48 MB 01:28 2023-07-14T19:15:32.830 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:15:32.830 INFO:teuthology.orchestra.run.smithi082.stdout:Total 2.1 MB/s | 206 MB 01:39 2023-07-14T19:15:33.082 INFO:teuthology.orchestra.run.smithi133.stdout:(34/131): ceph-volume-18.0.0-4940.g81c9387d.el8 307 kB/s | 258 kB 00:00 2023-07-14T19:15:33.235 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T19:15:33.528 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 115/134 2023-07-14T19:15:33.624 INFO:teuthology.orchestra.run.smithi133.stdout:(35/131): cephadm-18.0.0-4940.g81c9387d.el8.noa 305 kB/s | 165 kB 00:00 2023-07-14T19:15:33.636 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-07-14T19:15:33.662 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T19:15:33.741 INFO:teuthology.orchestra.run.smithi133.stdout:(36/131): cephfs-top-18.0.0-4940.g81c9387d.el8. 154 kB/s | 18 kB 00:00 2023-07-14T19:15:33.755 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-07-14T19:15:33.755 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-07-14T19:15:33.860 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T19:15:34.716 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T19:15:34.869 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-07-14T19:15:34.873 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-07-14T19:15:35.027 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T19:15:35.066 INFO:teuthology.orchestra.run.smithi133.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 231 kB/s | 306 kB 00:01 2023-07-14T19:15:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 119/134 2023-07-14T19:15:35.375 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T19:15:35.383 INFO:teuthology.orchestra.run.smithi133.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 186 kB/s | 59 kB 00:00 2023-07-14T19:15:35.533 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 121/134 2023-07-14T19:15:35.680 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T19:15:35.783 INFO:teuthology.orchestra.run.smithi133.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 188 kB/s | 75 kB 00:00 2023-07-14T19:15:37.804 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:15:38.045 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:15:38.064 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:15:38.204 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 124/134 2023-07-14T19:15:38.450 INFO:teuthology.orchestra.run.smithi133.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 178 kB/s | 476 kB 00:02 2023-07-14T19:15:38.495 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 125/134 2023-07-14T19:15:38.641 INFO:teuthology.orchestra.run.smithi133.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 227 kB/s | 44 kB 00:00 2023-07-14T19:15:38.792 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 126/134 2023-07-14T19:15:39.059 INFO:teuthology.orchestra.run.smithi133.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 444 kB/s | 185 kB 00:00 2023-07-14T19:15:39.072 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T19:15:39.167 INFO:teuthology.orchestra.run.smithi133.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-07-14T19:15:39.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T19:15:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T19:15:39.276 INFO:teuthology.orchestra.run.smithi133.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-07-14T19:15:39.385 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T19:15:39.561 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T19:15:39.601 INFO:teuthology.orchestra.run.smithi133.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 530 kB/s | 173 kB 00:00 2023-07-14T19:15:39.640 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-07-14T19:15:39.651 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T19:15:39.816 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T19:15:39.853 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T19:15:39.866 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T19:15:39.867 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:15:39.990 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T19:15:40.048 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:15:40.048 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T19:15:40.233 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-14T19:15:40.259 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T19:15:40.260 INFO:teuthology.orchestra.run.smithi133.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 584 kB/s | 384 kB 00:00 2023-07-14T19:15:40.401 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T19:15:40.452 INFO:teuthology.orchestra.run.smithi133.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 424 kB/s | 81 kB 00:00 2023-07-14T19:15:40.452 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:15:40.502 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-14T19:15:40.552 INFO:teuthology.orchestra.run.smithi133.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-07-14T19:15:40.644 INFO:teuthology.orchestra.run.smithi133.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 192 kB/s | 18 kB 00:00 2023-07-14T19:15:40.751 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T19:15:40.971 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:15:40.971 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 1/134 2023-07-14T19:15:40.971 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2/134 2023-07-14T19:15:40.971 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 3/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 4/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 5/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T19:15:40.972 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 10/134 2023-07-14T19:15:40.973 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 11/134 2023-07-14T19:15:40.973 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 12/134 2023-07-14T19:15:40.973 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 13/134 2023-07-14T19:15:40.973 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 14/134 2023-07-14T19:15:40.973 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 15/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 16/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 17/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 18/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 19/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 20/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 21/134 2023-07-14T19:15:40.974 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 22/134 2023-07-14T19:15:40.975 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 23/134 2023-07-14T19:15:40.975 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 24/134 2023-07-14T19:15:40.975 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 25/134 2023-07-14T19:15:40.975 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 26/134 2023-07-14T19:15:40.976 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 27/134 2023-07-14T19:15:40.976 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 28/134 2023-07-14T19:15:40.976 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 29/134 2023-07-14T19:15:40.976 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 30/134 2023-07-14T19:15:40.985 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 31/134 2023-07-14T19:15:40.985 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 32/134 2023-07-14T19:15:40.985 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 33/134 2023-07-14T19:15:40.986 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 34/134 2023-07-14T19:15:40.986 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 35/134 2023-07-14T19:15:40.986 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 36/134 2023-07-14T19:15:40.986 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 37/134 2023-07-14T19:15:40.987 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 38/134 2023-07-14T19:15:40.987 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T19:15:40.987 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T19:15:40.988 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T19:15:40.988 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T19:15:40.988 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T19:15:40.988 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T19:15:40.989 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T19:15:40.989 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T19:15:40.989 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T19:15:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T19:15:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T19:15:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T19:15:40.990 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T19:15:40.991 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T19:15:40.992 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T19:15:40.992 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T19:15:40.992 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T19:15:40.992 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T19:15:40.992 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T19:15:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T19:15:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T19:15:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T19:15:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T19:15:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T19:15:40.993 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T19:15:40.994 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T19:15:40.995 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T19:15:40.996 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T19:15:40.997 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T19:15:40.998 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T19:15:40.999 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T19:15:40.999 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T19:15:40.999 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T19:15:40.999 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T19:15:40.999 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T19:15:40.999 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T19:15:41.000 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T19:15:41.000 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T19:15:41.000 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T19:15:41.000 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T19:15:41.000 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T19:15:41.000 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T19:15:41.001 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T19:15:41.001 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T19:15:41.001 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T19:15:41.001 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T19:15:41.001 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T19:15:41.002 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T19:15:41.002 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T19:15:41.002 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T19:15:41.002 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T19:15:41.003 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T19:15:41.003 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T19:15:41.003 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T19:15:41.004 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 129/134 2023-07-14T19:15:41.004 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T19:15:41.004 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 131/134 2023-07-14T19:15:41.004 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:15:41.005 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T19:15:41.138 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-14T19:15:41.370 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:15:41.772 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:15:41.992 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:15:42.118 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:15:42.136 INFO:teuthology.orchestra.run.smithi133.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 800 kB/s | 1.2 MB 00:01 2023-07-14T19:15:42.319 INFO:teuthology.orchestra.run.smithi133.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 266 kB/s | 49 kB 00:00 2023-07-14T19:15:42.470 INFO:teuthology.orchestra.run.smithi133.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 392 kB/s | 59 kB 00:00 2023-07-14T19:15:42.475 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:15:42.573 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:15:42.628 INFO:teuthology.orchestra.run.smithi133.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 380 kB/s | 60 kB 00:00 2023-07-14T19:15:42.775 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T19:15:43.021 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T19:15:43.112 INFO:teuthology.orchestra.run.smithi133.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 585 kB/s | 283 kB 00:00 2023-07-14T19:15:43.220 INFO:teuthology.orchestra.run.smithi133.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-07-14T19:15:43.228 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T19:15:43.312 INFO:teuthology.orchestra.run.smithi133.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 367 kB/s | 34 kB 00:00 2023-07-14T19:15:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:15:43.408 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:58 ago on Fri 14 Jul 2023 07:13:45 PM UTC. 2023-07-14T19:15:43.604 INFO:teuthology.orchestra.run.smithi133.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 673 kB/s | 196 kB 00:00 2023-07-14T19:15:43.635 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T19:15:43.771 INFO:teuthology.orchestra.run.smithi133.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 369 kB/s | 61 kB 00:00 2023-07-14T19:15:43.880 INFO:teuthology.orchestra.run.smithi133.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 173 kB/s | 19 kB 00:00 2023-07-14T19:15:43.895 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T19:15:43.988 INFO:teuthology.orchestra.run.smithi133.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 221 kB/s | 24 kB 00:00 2023-07-14T19:15:44.075 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T19:15:44.097 INFO:teuthology.orchestra.run.smithi133.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 270 kB/s | 29 kB 00:00 2023-07-14T19:15:44.185 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T19:15:44.214 INFO:teuthology.orchestra.run.smithi133.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 236 kB/s | 27 kB 00:00 2023-07-14T19:15:44.333 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T19:15:44.412 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T19:15:44.550 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T19:15:44.673 INFO:teuthology.orchestra.run.smithi133.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 557 kB/s | 255 kB 00:00 2023-07-14T19:15:44.680 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-14T19:15:44.842 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T19:15:45.442 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-14T19:15:45.616 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T19:15:45.764 INFO:teuthology.orchestra.run.smithi133.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 230 kB/s | 251 kB 00:01 2023-07-14T19:15:45.998 INFO:teuthology.orchestra.run.smithi133.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 262 kB/s | 61 kB 00:00 2023-07-14T19:15:46.198 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-14T19:15:46.356 INFO:teuthology.orchestra.run.smithi133.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 238 kB/s | 85 kB 00:00 2023-07-14T19:15:46.391 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T19:15:46.499 INFO:teuthology.orchestra.run.smithi133.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 162 kB/s | 23 kB 00:00 2023-07-14T19:15:46.794 INFO:teuthology.orchestra.run.smithi161.stdout:Installed products updated. 2023-07-14T19:15:47.009 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:15:47.010 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2023-07-14T19:15:47.010 INFO:teuthology.orchestra.run.smithi161.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T19:15:47.010 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.010 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.010 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.011 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.012 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.012 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.012 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.012 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.012 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.012 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.013 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:15:47.014 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-3.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T19:15:47.015 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T19:15:47.016 INFO:teuthology.orchestra.run.smithi161.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.017 INFO:teuthology.orchestra.run.smithi161.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T19:15:47.018 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T19:15:47.019 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T19:15:47.020 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T19:15:47.021 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T19:15:47.022 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T19:15:47.023 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T19:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T19:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T19:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T19:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T19:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T19:15:47.024 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.025 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T19:15:47.026 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T19:15:47.027 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T19:15:47.028 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T19:15:47.028 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T19:15:47.028 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.028 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.029 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:15:47.029 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T19:15:47.029 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T19:15:47.029 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T19:15:47.030 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T19:15:47.030 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T19:15:47.031 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T19:15:47.031 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:15:47.032 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-07-14T19:15:47.059 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-14T19:15:47.194 DEBUG:teuthology.parallel:result is None 2023-07-14T19:15:47.306 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-14T19:15:47.525 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-14T19:15:47.744 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-14T19:15:47.980 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-14T19:15:48.196 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-14T19:15:48.442 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T19:15:48.606 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 26/134 2023-07-14T19:15:48.743 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 27/134 2023-07-14T19:15:49.623 INFO:teuthology.orchestra.run.smithi133.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 568 kB/s | 1.7 MB 00:03 2023-07-14T19:15:49.648 INFO:teuthology.orchestra.run.smithi133.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-14T19:15:49.664 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 28/134 2023-07-14T19:15:49.707 INFO:teuthology.orchestra.run.smithi133.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2023-07-14T19:15:49.741 INFO:teuthology.orchestra.run.smithi133.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 5.9 MB/s | 200 kB 00:00 2023-07-14T19:15:49.775 INFO:teuthology.orchestra.run.smithi133.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-07-14T19:15:49.817 INFO:teuthology.orchestra.run.smithi133.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-14T19:15:49.859 INFO:teuthology.orchestra.run.smithi133.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-07-14T19:15:49.901 INFO:teuthology.orchestra.run.smithi133.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-14T19:15:49.943 INFO:teuthology.orchestra.run.smithi133.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-14T19:15:49.979 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-14T19:15:49.984 INFO:teuthology.orchestra.run.smithi133.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-07-14T19:15:50.026 INFO:teuthology.orchestra.run.smithi133.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-07-14T19:15:50.068 INFO:teuthology.orchestra.run.smithi133.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 13 MB/s | 560 kB 00:00 2023-07-14T19:15:50.094 INFO:teuthology.orchestra.run.smithi133.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 746 kB/s | 19 kB 00:00 2023-07-14T19:15:50.136 INFO:teuthology.orchestra.run.smithi133.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-07-14T19:15:50.178 INFO:teuthology.orchestra.run.smithi133.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 13 MB/s | 559 kB 00:00 2023-07-14T19:15:50.203 INFO:teuthology.orchestra.run.smithi133.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 717 kB/s | 18 kB 00:00 2023-07-14T19:15:50.241 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-14T19:15:50.245 INFO:teuthology.orchestra.run.smithi133.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 14 MB/s | 605 kB 00:00 2023-07-14T19:15:50.295 INFO:teuthology.orchestra.run.smithi133.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-07-14T19:15:50.379 INFO:teuthology.orchestra.run.smithi133.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 28 MB/s | 2.3 MB 00:00 2023-07-14T19:15:50.421 INFO:teuthology.orchestra.run.smithi133.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-14T19:15:50.454 INFO:teuthology.orchestra.run.smithi133.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-07-14T19:15:50.496 INFO:teuthology.orchestra.run.smithi133.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-07-14T19:15:50.527 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-14T19:15:50.538 INFO:teuthology.orchestra.run.smithi133.stdout:(90/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-07-14T19:15:50.580 INFO:teuthology.orchestra.run.smithi133.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2023-07-14T19:15:50.622 INFO:teuthology.orchestra.run.smithi133.stdout:(92/131): libcom_err-devel-1.45.6-4.el8.x86_64. 925 kB/s | 39 kB 00:00 2023-07-14T19:15:50.672 INFO:teuthology.orchestra.run.smithi133.stdout:(93/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 13 MB/s | 644 kB 00:00 2023-07-14T19:15:50.715 INFO:teuthology.orchestra.run.smithi133.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 171 kB 00:00 2023-07-14T19:15:50.724 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T19:15:50.748 INFO:teuthology.orchestra.run.smithi133.stdout:(95/131): NetworkManager-initscripts-updown-1.3 4.0 MB/s | 138 kB 00:00 2023-07-14T19:15:51.023 INFO:teuthology.orchestra.run.smithi133.stdout:(96/131): python3-scipy-1.0.0-21.module+el8.5.0 50 MB/s | 14 MB 00:00 2023-07-14T19:15:51.057 INFO:teuthology.orchestra.run.smithi133.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.2 MB/s | 110 kB 00:00 2023-07-14T19:15:51.090 INFO:teuthology.orchestra.run.smithi133.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-14T19:15:51.133 INFO:teuthology.orchestra.run.smithi133.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-14T19:15:51.175 INFO:teuthology.orchestra.run.smithi133.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-14T19:15:51.217 INFO:teuthology.orchestra.run.smithi133.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 930 kB/s | 39 kB 00:00 2023-07-14T19:15:51.267 INFO:teuthology.orchestra.run.smithi133.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-14T19:15:51.317 INFO:teuthology.orchestra.run.smithi133.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 10 MB/s | 538 kB 00:00 2023-07-14T19:15:51.368 INFO:teuthology.orchestra.run.smithi133.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-07-14T19:15:51.410 INFO:teuthology.orchestra.run.smithi133.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-14T19:15:51.460 INFO:teuthology.orchestra.run.smithi133.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-14T19:15:51.577 INFO:teuthology.orchestra.run.smithi133.stdout:(107/131): python3-babel-2.5.1-7.el8.noarch.rpm 41 MB/s | 4.8 MB 00:00 2023-07-14T19:15:51.618 INFO:teuthology.orchestra.run.smithi133.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 639 kB/s | 27 kB 00:00 2023-07-14T19:15:51.661 INFO:teuthology.orchestra.run.smithi133.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-14T19:15:51.702 INFO:teuthology.orchestra.run.smithi133.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-07-14T19:15:51.811 INFO:teuthology.orchestra.run.smithi133.stdout:(111/131): python3-numpy-1.14.3-10.el8.x86_64.r 35 MB/s | 3.7 MB 00:00 2023-07-14T19:15:51.861 INFO:teuthology.orchestra.run.smithi133.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-14T19:15:51.903 INFO:teuthology.orchestra.run.smithi133.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-14T19:15:51.945 INFO:teuthology.orchestra.run.smithi133.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-14T19:15:51.987 INFO:teuthology.orchestra.run.smithi133.stdout:(115/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-07-14T19:15:52.029 INFO:teuthology.orchestra.run.smithi133.stdout:(116/131): python3-jsonpointer-1.10-11.el8.noar 478 kB/s | 20 kB 00:00 2023-07-14T19:15:52.071 INFO:teuthology.orchestra.run.smithi133.stdout:(117/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-07-14T19:15:52.130 INFO:teuthology.orchestra.run.smithi133.stdout:(118/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-14T19:15:52.246 INFO:teuthology.orchestra.run.smithi133.stdout:(119/131): openblas-threads-0.3.15-3.el8.x86_64 42 MB/s | 4.9 MB 00:00 2023-07-14T19:15:52.288 INFO:teuthology.orchestra.run.smithi133.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-14T19:15:52.405 INFO:teuthology.orchestra.run.smithi133.stdout:(121/131): openblas-0.3.15-3.el8.x86_64.rpm 40 MB/s | 4.7 MB 00:00 2023-07-14T19:15:52.447 INFO:teuthology.orchestra.run.smithi133.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-14T19:15:52.484 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-14T19:15:52.489 INFO:teuthology.orchestra.run.smithi133.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-14T19:15:52.606 INFO:teuthology.orchestra.run.smithi133.stdout:(124/131): postgresql-server-10.21-2.module+el8 43 MB/s | 5.1 MB 00:00 2023-07-14T19:15:52.665 INFO:teuthology.orchestra.run.smithi133.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-07-14T19:15:52.723 INFO:teuthology.orchestra.run.smithi133.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 26 MB/s | 1.5 MB 00:00 2023-07-14T19:15:52.842 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-14T19:15:52.865 INFO:teuthology.orchestra.run.smithi133.stdout:(127/131): ceph-mgr-diskprediction-local-18.0.0 345 kB/s | 7.4 MB 00:22 2023-07-14T19:15:53.068 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T19:15:53.292 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T19:15:53.479 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 36/134 2023-07-14T19:15:53.652 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 37/134 2023-07-14T19:15:53.814 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-14T19:15:53.950 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-14T19:15:54.150 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-14T19:15:54.344 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-14T19:15:54.606 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-14T19:15:54.829 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-14T19:15:55.213 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-14T19:15:55.509 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-14T19:15:55.732 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-14T19:15:55.950 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-14T19:15:56.193 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-14T19:15:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-14T19:15:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-14T19:15:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-14T19:15:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-14T19:15:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-14T19:15:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-14T19:15:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-14T19:15:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-14T19:15:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-14T19:15:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-14T19:15:58.637 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-14T19:15:58.797 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-14T19:15:58.971 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T19:15:59.040 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-14T19:15:59.309 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-14T19:15:59.491 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-14T19:15:59.647 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T19:15:59.711 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-14T19:15:59.934 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-14T19:16:00.114 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-14T19:16:00.271 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T19:16:00.449 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T19:16:00.713 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T19:16:00.928 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T19:16:01.075 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T19:16:01.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T19:16:01.462 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T19:16:01.640 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T19:16:01.806 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T19:16:01.982 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T19:16:02.135 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T19:16:02.355 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T19:16:02.565 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T19:16:02.760 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T19:16:03.006 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T19:16:03.278 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T19:16:03.461 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T19:16:03.679 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T19:16:04.764 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T19:16:05.049 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T19:16:05.311 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T19:16:05.629 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T19:16:05.922 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T19:16:05.972 INFO:teuthology.orchestra.run.smithi133.stdout:(128/131): librados2-18.0.0-4940.g81c9387d.el8. 284 kB/s | 3.7 MB 00:13 2023-07-14T19:16:06.023 INFO:teuthology.orchestra.run.smithi133.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-14T19:16:06.106 INFO:teuthology.orchestra.run.smithi133.stdout:(130/131): librbd1-18.0.0-4940.g81c9387d.el8.x8 281 kB/s | 3.6 MB 00:13 2023-07-14T19:16:06.134 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T19:16:06.905 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T19:16:07.111 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T19:16:07.171 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T19:16:07.337 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 93/134 2023-07-14T19:16:11.011 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:16:11.219 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:16:11.806 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:16:12.185 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T19:16:12.255 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T19:16:12.410 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T19:16:31.056 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T19:16:31.056 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2023-07-14T19:16:31.056 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /run 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2023-07-14T19:16:31.057 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-07-14T19:16:31.058 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:16:31.406 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T19:16:32.982 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T19:16:33.188 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T19:16:33.408 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T19:16:33.614 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T19:16:34.068 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T19:16:34.256 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T19:16:34.272 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T19:16:35.925 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T19:16:36.082 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T19:16:36.195 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 102/134 2023-07-14T19:16:36.386 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 103/134 2023-07-14T19:16:36.580 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T19:16:36.792 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T19:16:37.088 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T19:16:38.045 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T19:16:38.205 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T19:16:38.295 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T19:16:38.648 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 107/134 2023-07-14T19:16:38.864 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T19:16:38.899 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T19:16:39.063 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T19:16:39.074 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T19:16:40.403 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 110/134 2023-07-14T19:16:40.624 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T19:16:40.659 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T19:16:40.804 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T19:16:40.845 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T19:16:40.987 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T19:16:41.339 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T19:16:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T19:16:46.050 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T19:16:46.329 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 115/134 2023-07-14T19:16:46.478 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T19:16:46.672 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-14T19:16:47.561 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-14T19:16:47.859 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-14T19:16:48.030 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 119/134 2023-07-14T19:16:48.248 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T19:16:48.414 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 121/134 2023-07-14T19:16:48.562 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T19:16:50.762 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:16:51.010 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:16:51.026 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:16:51.194 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 124/134 2023-07-14T19:16:51.476 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 125/134 2023-07-14T19:16:51.761 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 126/134 2023-07-14T19:16:52.053 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T19:16:52.210 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T19:16:52.218 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T19:16:52.409 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-14T19:16:52.618 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T19:16:52.697 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-14T19:16:52.926 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T19:16:52.938 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-14T19:16:52.938 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:16:53.137 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:16:53.138 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T19:16:53.398 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T19:16:53.608 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:16:54.147 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:16:54.147 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 1/134 2023-07-14T19:16:54.147 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2/134 2023-07-14T19:16:54.148 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 3/134 2023-07-14T19:16:54.148 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 4/134 2023-07-14T19:16:54.148 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 5/134 2023-07-14T19:16:54.149 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:16:54.149 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:16:54.149 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:16:54.150 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T19:16:54.150 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 10/134 2023-07-14T19:16:54.150 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 11/134 2023-07-14T19:16:54.150 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 12/134 2023-07-14T19:16:54.151 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 13/134 2023-07-14T19:16:54.151 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 14/134 2023-07-14T19:16:54.151 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 15/134 2023-07-14T19:16:54.151 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 16/134 2023-07-14T19:16:54.152 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 17/134 2023-07-14T19:16:54.152 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 18/134 2023-07-14T19:16:54.152 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 19/134 2023-07-14T19:16:54.153 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 20/134 2023-07-14T19:16:54.153 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 21/134 2023-07-14T19:16:54.163 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 22/134 2023-07-14T19:16:54.163 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 23/134 2023-07-14T19:16:54.163 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 24/134 2023-07-14T19:16:54.164 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 25/134 2023-07-14T19:16:54.164 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 26/134 2023-07-14T19:16:54.164 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 27/134 2023-07-14T19:16:54.164 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 28/134 2023-07-14T19:16:54.165 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 29/134 2023-07-14T19:16:54.165 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 30/134 2023-07-14T19:16:54.165 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 31/134 2023-07-14T19:16:54.165 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 32/134 2023-07-14T19:16:54.166 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 33/134 2023-07-14T19:16:54.166 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 34/134 2023-07-14T19:16:54.166 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 35/134 2023-07-14T19:16:54.167 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 36/134 2023-07-14T19:16:54.167 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 37/134 2023-07-14T19:16:54.167 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 38/134 2023-07-14T19:16:54.168 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T19:16:54.168 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T19:16:54.168 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T19:16:54.169 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T19:16:54.169 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T19:16:54.169 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T19:16:54.169 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T19:16:54.170 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T19:16:54.170 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T19:16:54.170 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T19:16:54.170 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T19:16:54.171 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T19:16:54.171 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T19:16:54.171 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T19:16:54.171 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T19:16:54.172 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T19:16:54.172 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T19:16:54.172 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T19:16:54.172 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T19:16:54.173 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T19:16:54.173 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T19:16:54.173 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T19:16:54.173 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T19:16:54.174 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T19:16:54.174 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T19:16:54.174 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T19:16:54.175 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-14T19:16:54.176 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-14T19:16:54.177 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-14T19:16:54.178 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-14T19:16:54.179 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-14T19:16:54.180 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-14T19:16:54.181 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-14T19:16:54.182 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 129/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T19:16:54.183 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 131/134 2023-07-14T19:16:54.184 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:16:54.184 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T19:16:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:16:56.628 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:03:15 ago on Fri 14 Jul 2023 07:13:41 PM UTC. 2023-07-14T19:16:59.252 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-07-14T19:16:59.468 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:16:59.469 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2023-07-14T19:16:59.469 INFO:teuthology.orchestra.run.smithi082.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T19:16:59.469 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.469 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.469 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-07-14T19:16:59.470 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T19:16:59.470 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T19:16:59.470 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T19:16:59.470 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.470 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.470 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.471 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.472 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T19:16:59.473 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T19:16:59.474 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T19:16:59.474 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T19:16:59.474 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-3.el8.x86_64 2023-07-14T19:16:59.474 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T19:16:59.474 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T19:16:59.474 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T19:16:59.475 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.475 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.475 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.476 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T19:16:59.476 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T19:16:59.476 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T19:16:59.477 INFO:teuthology.orchestra.run.smithi082.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T19:16:59.477 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T19:16:59.477 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T19:16:59.477 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.478 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.478 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T19:16:59.478 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.479 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T19:16:59.479 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T19:16:59.479 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T19:16:59.479 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T19:16:59.480 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T19:16:59.480 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T19:16:59.480 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T19:16:59.480 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T19:16:59.481 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T19:16:59.481 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T19:16:59.481 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T19:16:59.482 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T19:16:59.482 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T19:16:59.482 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T19:16:59.482 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:16:59.483 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:16:59.483 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:16:59.483 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T19:16:59.483 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T19:16:59.484 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T19:16:59.484 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T19:16:59.484 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T19:16:59.485 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.485 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.485 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.485 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T19:16:59.486 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T19:16:59.486 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T19:16:59.486 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T19:16:59.487 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T19:16:59.487 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T19:16:59.487 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T19:16:59.487 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T19:16:59.488 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T19:16:59.488 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T19:16:59.488 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T19:16:59.489 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T19:16:59.489 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T19:16:59.489 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T19:16:59.489 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T19:16:59.490 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T19:16:59.490 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T19:16:59.490 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T19:16:59.490 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T19:16:59.491 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T19:16:59.491 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T19:16:59.491 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T19:16:59.491 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T19:16:59.492 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T19:16:59.492 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T19:16:59.493 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T19:16:59.493 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T19:16:59.493 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T19:16:59.494 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T19:16:59.494 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.494 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.494 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T19:16:59.495 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T19:16:59.495 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.495 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T19:16:59.496 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T19:16:59.496 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T19:16:59.496 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T19:16:59.496 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T19:16:59.497 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T19:16:59.497 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T19:16:59.497 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T19:16:59.498 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T19:16:59.498 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T19:16:59.498 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T19:16:59.498 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T19:16:59.499 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T19:16:59.499 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.499 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.499 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:16:59.499 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T19:16:59.500 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T19:16:59.500 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T19:16:59.500 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T19:16:59.500 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T19:16:59.501 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T19:16:59.501 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:16:59.501 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-07-14T19:16:59.658 DEBUG:teuthology.parallel:result is None 2023-07-14T19:17:41.700 INFO:teuthology.orchestra.run.smithi133.stdout:(131/131): ceph-test-18.0.0-4940.g81c9387d.el8. 318 kB/s | 48 MB 02:34 2023-07-14T19:17:41.703 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:17:41.703 INFO:teuthology.orchestra.run.smithi133.stdout:Total 972 kB/s | 206 MB 03:36 2023-07-14T19:17:42.473 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-07-14T19:17:42.592 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-07-14T19:17:42.592 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-07-14T19:17:43.834 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-07-14T19:17:43.838 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-07-14T19:17:48.608 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-07-14T19:17:48.772 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-14T19:17:48.946 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-14T19:17:49.116 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T19:17:49.210 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-14T19:17:49.424 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T19:17:49.834 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-14T19:17:50.024 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 5/134 2023-07-14T19:17:50.417 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 5/134 2023-07-14T19:17:50.624 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:17:50.757 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:17:50.924 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:17:51.021 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:17:51.170 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:17:51.341 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-14T19:17:51.601 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-14T19:17:51.801 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-14T19:17:52.180 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-14T19:17:52.402 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-14T19:17:52.573 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T19:17:52.677 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 14/134 2023-07-14T19:17:52.807 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T19:17:52.942 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 15/134 2023-07-14T19:17:53.160 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-14T19:17:53.329 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-14T19:17:53.513 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-14T19:17:53.648 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T19:17:53.785 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-14T19:17:53.948 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T19:17:54.555 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-14T19:17:54.715 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T19:17:55.321 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-14T19:17:55.489 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T19:17:56.180 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-14T19:17:56.438 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-14T19:17:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-14T19:17:56.874 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-14T19:17:57.071 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 26/134 2023-07-14T19:17:57.209 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 27/134 2023-07-14T19:17:57.434 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 28/134 2023-07-14T19:17:57.631 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T19:17:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-14T19:18:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-14T19:18:00.686 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-14T19:18:00.975 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-14T19:18:01.186 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-14T19:18:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-14T19:18:01.599 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-14T19:18:01.769 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 36/134 2023-07-14T19:18:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 37/134 2023-07-14T19:18:02.079 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-14T19:18:02.230 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-14T19:18:02.405 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T19:18:02.474 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-14T19:18:02.681 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-14T19:18:02.925 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-14T19:18:03.063 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T19:18:03.144 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-14T19:18:03.350 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-14T19:18:03.500 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-14T19:18:03.661 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-14T19:18:03.823 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-14T19:18:03.991 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-14T19:18:04.132 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-14T19:18:04.350 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-14T19:18:04.532 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-14T19:18:04.894 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-14T19:18:05.190 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-14T19:18:05.410 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-14T19:18:05.642 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-14T19:18:05.834 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-14T19:18:06.129 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-14T19:18:06.321 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-14T19:18:06.505 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-14T19:18:06.758 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-14T19:18:06.957 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-14T19:18:07.115 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-14T19:18:07.736 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-14T19:18:07.982 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-14T19:18:08.208 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-14T19:18:08.414 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-14T19:18:08.570 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-14T19:18:08.765 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-14T19:18:09.046 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-14T19:18:09.269 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-14T19:18:09.433 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-14T19:18:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-14T19:18:09.795 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-14T19:18:09.997 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-14T19:18:10.180 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-14T19:18:10.356 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-14T19:18:10.492 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-14T19:18:10.704 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-14T19:18:10.916 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-14T19:18:11.126 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-14T19:18:11.359 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-14T19:18:11.577 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-14T19:18:11.750 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-14T19:18:11.974 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-14T19:18:13.064 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-14T19:18:13.348 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-14T19:18:13.611 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-14T19:18:13.920 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-14T19:18:14.206 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-14T19:18:14.410 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-14T19:18:15.186 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-14T19:18:15.328 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T19:18:15.406 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 92/134 2023-07-14T19:18:15.561 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 93/134 2023-07-14T19:18:19.152 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:18:19.403 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:18:19.981 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 94/134 2023-07-14T19:18:20.232 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T19:18:20.299 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 95/134 2023-07-14T19:18:20.435 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T19:18:39.080 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 96/134 2023-07-14T19:18:39.080 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2023-07-14T19:18:39.080 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2023-07-14T19:18:39.080 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /run 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-07-14T19:18:39.081 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:18:39.416 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T19:18:40.991 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 97/134 2023-07-14T19:18:41.186 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T19:18:41.405 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 98/134 2023-07-14T19:18:41.582 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T19:18:42.034 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 99/134 2023-07-14T19:18:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T19:18:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 100/134 2023-07-14T19:18:43.847 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T19:18:44.022 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 101/134 2023-07-14T19:18:44.171 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 102/134 2023-07-14T19:18:44.402 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 103/134 2023-07-14T19:18:44.597 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T19:18:44.830 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 104/134 2023-07-14T19:18:45.110 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T19:18:46.066 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 105/134 2023-07-14T19:18:46.248 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T19:18:46.338 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 106/134 2023-07-14T19:18:46.680 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 107/134 2023-07-14T19:18:46.881 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T19:18:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 108/134 2023-07-14T19:18:47.072 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T19:18:47.083 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 109/134 2023-07-14T19:18:48.389 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 110/134 2023-07-14T19:18:48.596 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T19:18:48.630 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 111/134 2023-07-14T19:18:48.781 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T19:18:48.822 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4940.g81c93 112/134 2023-07-14T19:18:48.980 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T19:18:49.331 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 113/134 2023-07-14T19:18:49.506 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T19:18:54.031 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 114/134 2023-07-14T19:18:54.881 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 115/134 2023-07-14T19:18:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-14T19:18:55.357 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 117/134 2023-07-14T19:18:55.616 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-14T19:18:55.762 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T19:18:55.961 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-14T19:18:56.192 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-14T19:18:56.399 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 121/134 2023-07-14T19:18:56.538 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-14T19:18:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:18:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:18:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-14T19:18:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 124/134 2023-07-14T19:18:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 125/134 2023-07-14T19:18:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 126/134 2023-07-14T19:18:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-14T19:19:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T19:19:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-14T19:19:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T19:19:00.440 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-14T19:19:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T19:19:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-14T19:19:00.756 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-14T19:19:00.756 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:19:00.965 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:19:00.965 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T19:19:01.202 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-14T19:19:01.410 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:19:01.908 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:19:01.908 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 1/134 2023-07-14T19:19:01.908 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2/134 2023-07-14T19:19:01.908 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 3/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 4/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4940.g81c93 5/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 6/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 7/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 8/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 9/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 10/134 2023-07-14T19:19:01.909 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 11/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 12/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 13/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_ 14/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 15/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 16/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_6 17/134 2023-07-14T19:19:01.910 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86 18/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 19/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el 20/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:18.0.0-4940.g81c9387d.el8. 21/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_6 22/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 23/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 24/134 2023-07-14T19:19:01.911 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 25/134 2023-07-14T19:19:01.918 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 26/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 27/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 28/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d. 29/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noa 30/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.n 31/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c 32/134 2023-07-14T19:19:01.919 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el 33/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 34/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.e 35/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 36/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 37/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 38/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-14T19:19:01.920 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-14T19:19:01.921 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-14T19:19:01.922 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-14T19:19:01.923 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-14T19:19:01.924 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-14T19:19:01.925 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-14T19:19:01.925 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-14T19:19:01.925 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-14T19:19:01.925 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-14T19:19:01.925 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-14T19:19:01.925 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-14T19:19:01.926 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-14T19:19:01.926 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-14T19:19:01.926 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-14T19:19:01.926 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-14T19:19:01.926 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-14T19:19:01.927 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-14T19:19:01.928 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-14T19:19:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-14T19:19:01.930 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-14T19:19:01.931 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-14T19:19:01.932 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 129/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 131/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-14T19:19:01.933 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-14T19:19:04.365 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-14T19:19:04.366 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:05:11 ago on Fri 14 Jul 2023 07:13:53 PM UTC. 2023-07-14T19:19:06.956 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-07-14T19:19:07.169 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:19:07.170 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2023-07-14T19:19:07.170 INFO:teuthology.orchestra.run.smithi133.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-14T19:19:07.170 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.170 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.170 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-07-14T19:19:07.170 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-14T19:19:07.171 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.1-7.el8.noarch 2023-07-14T19:19:07.171 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.0.4-10.el8.x86_64 2023-07-14T19:19:07.171 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.171 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.171 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.172 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.173 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:18.0.0-4940.g81c9387d.el8.noarch 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-14T19:19:07.174 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.1-9.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-3.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.175 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: libpq-13.5-1.el8.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-14T19:19:07.176 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-14T19:19:07.177 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.177 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.177 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-14T19:19:07.177 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.177 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-14T19:19:07.178 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:19:07.179 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-14T19:19:07.180 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-14T19:19:07.181 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-14T19:19:07.182 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-14T19:19:07.183 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-14T19:19:07.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-2.6-1.el8.noarch 2023-07-14T19:19:07.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-14T19:19:07.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-14T19:19:07.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-14T19:19:07.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-14T19:19:07.184 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-14T19:19:07.185 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-14T19:19:07.186 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-14T19:19:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-14T19:19:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-14T19:19:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-14T19:19:07.187 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:18.0.0-4940.g81c9387d.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-14T19:19:07.188 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-14T19:19:07.189 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-14T19:19:07.189 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-07-14T19:19:07.189 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-07-14T19:19:07.374 DEBUG:teuthology.parallel:result is None 2023-07-14T19:19:07.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:19:07.890 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T19:19:07.920 INFO:teuthology.orchestra.run.smithi082.stdout:18.0.0-4940.g81c9387d.el8 2023-07-14T19:19:07.920 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4940.g81c9387d.el8 2023-07-14T19:19:07.921 INFO:teuthology.task.install:The correct ceph version 18.0.0-4940.g81c9387d is installed. 2023-07-14T19:19:07.921 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:19:08.423 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T19:19:08.451 INFO:teuthology.orchestra.run.smithi133.stdout:18.0.0-4940.g81c9387d.el8 2023-07-14T19:19:08.452 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4940.g81c9387d.el8 2023-07-14T19:19:08.452 INFO:teuthology.task.install:The correct ceph version 18.0.0-4940.g81c9387d is installed. 2023-07-14T19:19:08.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:19:08.884 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-14T19:19:08.913 INFO:teuthology.orchestra.run.smithi161.stdout:18.0.0-4940.g81c9387d.el8 2023-07-14T19:19:08.913 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4940.g81c9387d.el8 2023-07-14T19:19:08.914 INFO:teuthology.task.install:The correct ceph version 18.0.0-4940.g81c9387d is installed. 2023-07-14T19:19:08.915 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-14T19:19:08.915 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:19:08.916 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T19:19:08.964 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:19:08.965 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T19:19:09.012 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:19:09.013 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-14T19:19:09.061 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-14T19:19:09.062 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:19:09.062 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T19:19:09.093 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T19:19:09.165 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:19:09.165 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T19:19:09.199 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T19:19:09.272 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:19:09.273 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2023-07-14T19:19:09.305 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-14T19:19:09.375 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-14T19:19:09.375 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:19:09.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T19:19:09.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T19:19:09.475 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:19:09.475 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T19:19:09.507 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T19:19:09.577 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:19:09.578 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-14T19:19:09.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-14T19:19:09.679 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-14T19:19:09.679 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:19:09.680 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T19:19:09.710 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T19:19:09.780 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:19:09.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T19:19:09.811 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T19:19:09.881 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:19:09.882 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2023-07-14T19:19:09.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-14T19:19:09.986 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-14T19:19:10.078 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'subvols': {'create': 2}} 2023-07-14T19:19:10.079 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:19:10.089 INFO:tasks.cephadm:Cluster fsid is 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:19:10.091 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-14T19:19:10.092 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.82', 'mon.b': '172.21.15.133', 'mon.c': '172.21.15.161'} 2023-07-14T19:19:10.093 INFO:tasks.cephadm:First mon is mon.a on smithi082 2023-07-14T19:19:10.093 INFO:tasks.cephadm:First mgr is x 2023-07-14T19:19:10.093 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-14T19:19:10.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2023-07-14T19:19:10.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2023-07-14T19:19:10.160 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2023-07-14T19:19:10.192 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-14T19:19:10.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:19:15.676 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '81c9387d2e7576da065747ec9c16d1c7d69016ec', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71431/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4940-g81c9387d', 'node_name': '172.21.4.65+confusa03', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4940.g81c9387d'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-14 16:50:26.249583', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20230714.134335', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2023-07-14T19:19:16.265 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20230714.134335, sha1 81c9387d2e7576da065747ec9c16d1c7d69016ec from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:19:16.266 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm 2023-07-14T19:19:16.266 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:19.487 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-rw-r--. 1 ubuntu ubuntu 670007 Jul 14 19:19 /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:19.488 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:23.319 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-rw-r--. 1 ubuntu ubuntu 670007 Jul 14 19:19 /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:23.320 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20230714.134335/81c9387d2e7576da065747ec9c16d1c7d69016ec/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:26.420 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-rw-r--. 1 ubuntu ubuntu 670007 Jul 14 19:19 /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:26.420 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:26.442 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:26.464 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-14T19:19:26.518 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec on all hosts... 2023-07-14T19:19:26.519 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec pull 2023-07-14T19:19:26.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec pull 2023-07-14T19:19:26.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec pull 2023-07-14T19:19:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T19:19:27.787 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T19:19:29.297 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T19:19:57.027 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:19:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T19:19:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705", 2023-07-14T19:19:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2023-07-14T19:19:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1302190083259ae87fcd27babf423c1dd04ae7417edb252031c1d861adeb3110" 2023-07-14T19:19:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:19:57.029 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:19:57.897 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T19:19:57.898 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T19:19:57.898 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705", 2023-07-14T19:19:57.898 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2023-07-14T19:19:57.898 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1302190083259ae87fcd27babf423c1dd04ae7417edb252031c1d861adeb3110" 2023-07-14T19:19:57.898 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-07-14T19:19:57.898 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T19:20:03.406 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T19:20:03.407 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev)", 2023-07-14T19:20:03.407 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705", 2023-07-14T19:20:03.407 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2023-07-14T19:20:03.407 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1302190083259ae87fcd27babf423c1dd04ae7417edb252031c1d861adeb3110" 2023-07-14T19:20:03.407 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2023-07-14T19:20:03.407 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T19:20:03.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2023-07-14T19:20:03.466 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2023-07-14T19:20:03.504 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2023-07-14T19:20:03.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2023-07-14T19:20:03.566 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2023-07-14T19:20:03.598 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2023-07-14T19:20:03.626 INFO:tasks.cephadm:Writing seed config... 2023-07-14T19:20:03.628 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-14T19:20:03.628 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-14T19:20:03.628 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-14T19:20:03.629 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-14T19:20:03.629 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-14T19:20:03.629 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-14T19:20:03.629 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-14T19:20:03.630 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-14T19:20:03.630 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-14T19:20:03.630 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-14T19:20:03.630 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-14T19:20:03.630 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-14T19:20:03.631 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-14T19:20:03.631 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-14T19:20:03.631 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-14T19:20:03.631 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-14T19:20:03.631 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-14T19:20:03.632 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-14T19:20:03.632 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-14T19:20:03.632 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-14T19:20:03.632 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-14T19:20:03.633 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-14T19:20:03.633 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-14T19:20:03.633 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-14T19:20:03.633 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-14T19:20:03.633 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-14T19:20:03.634 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-14T19:20:03.634 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-14T19:20:03.635 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-14T19:20:03.635 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-14T19:20:03.636 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-14T19:20:03.636 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-14T19:20:03.637 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-14T19:20:03.637 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-14T19:20:03.638 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-14T19:20:03.638 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-14T19:20:03.639 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-14T19:20:03.639 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-14T19:20:03.640 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-14T19:20:03.640 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-14T19:20:03.641 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-14T19:20:03.641 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-14T19:20:03.642 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-14T19:20:03.642 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-14T19:20:03.644 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:20:03.644 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-14T19:20:03.665 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4f3eb35e-227b-11ee-9b33-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-14T19:20:03.665 DEBUG:teuthology.orchestra.run.smithi082:mon.a> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a.service 2023-07-14T19:20:03.707 DEBUG:teuthology.orchestra.run.smithi082:mgr.x> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x.service 2023-07-14T19:20:03.738 INFO:journalctl@ceph.mon.a.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:20:03.739 INFO:tasks.cephadm:Bootstrapping... 2023-07-14T19:20:03.740 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec -v bootstrap --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.82 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:20:03.768 INFO:journalctl@ceph.mgr.x.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:20:03.905 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-07-14T19:20:03.905 INFO:teuthology.orchestra.run.smithi082.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec', '-v', 'bootstrap', '--fsid', '4f3eb35e-227b-11ee-9b33-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.82', '--skip-admin-label'] 2023-07-14T19:20:04.054 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.1.1 2023-07-14T19:20:04.055 INFO:teuthology.orchestra.run.smithi082.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-14T19:20:04.055 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying podman|docker is present... 2023-07-14T19:20:04.198 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.1.1 2023-07-14T19:20:04.198 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying lvm2 is present... 2023-07-14T19:20:04.198 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying time synchronization is in place... 2023-07-14T19:20:04.205 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T19:20:04.206 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T19:20:04.211 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T19:20:04.211 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-07-14T19:20:04.217 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2023-07-14T19:20:04.223 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2023-07-14T19:20:04.223 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2023-07-14T19:20:04.224 INFO:teuthology.orchestra.run.smithi082.stdout:Repeating the final host check... 2023-07-14T19:20:04.364 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 4.1.1 2023-07-14T19:20:04.364 INFO:teuthology.orchestra.run.smithi082.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-14T19:20:04.364 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl is present 2023-07-14T19:20:04.364 INFO:teuthology.orchestra.run.smithi082.stdout:lvcreate is present 2023-07-14T19:20:04.370 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-14T19:20:04.370 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-14T19:20:04.376 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-14T19:20:04.376 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-07-14T19:20:04.382 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout enabled 2023-07-14T19:20:04.387 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout active 2023-07-14T19:20:04.387 INFO:teuthology.orchestra.run.smithi082.stdout:Unit chronyd.service is enabled and running 2023-07-14T19:20:04.388 INFO:teuthology.orchestra.run.smithi082.stdout:Host looks OK 2023-07-14T19:20:04.388 INFO:teuthology.orchestra.run.smithi082.stdout:Cluster fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:04.388 INFO:teuthology.orchestra.run.smithi082.stdout:Acquiring lock 140185303681512 on /run/cephadm/4f3eb35e-227b-11ee-9b33-001a4aab830c.lock 2023-07-14T19:20:04.388 INFO:teuthology.orchestra.run.smithi082.stdout:Lock 140185303681512 acquired on /run/cephadm/4f3eb35e-227b-11ee-9b33-001a4aab830c.lock 2023-07-14T19:20:04.389 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 3300 ... 2023-07-14T19:20:04.389 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying IP 172.21.15.82 port 6789 ... 2023-07-14T19:20:04.389 INFO:teuthology.orchestra.run.smithi082.stdout:Base mon IP(s) is [172.21.15.82:3300, 172.21.15.82:6789], mon addrv is [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2023-07-14T19:20:04.392 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-14T19:20:04.392 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.82 metric 100 2023-07-14T19:20:04.394 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-14T19:20:04.394 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-14T19:20:04.395 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-14T19:20:04.397 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-14T19:20:04.397 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-14T19:20:04.397 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T19:20:04.397 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-14T19:20:04.397 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout inet6 fe80::1e0c:ee15:690e:98ae/64 scope link noprefixroute 2023-07-14T19:20:04.397 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-14T19:20:04.398 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2023-07-14T19:20:04.398 INFO:teuthology.orchestra.run.smithi082.stdout:Mon IP `172.21.15.82` is in CIDR network `172.21.0.0/20` 2023-07-14T19:20:04.398 INFO:teuthology.orchestra.run.smithi082.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-14T19:20:04.399 INFO:teuthology.orchestra.run.smithi082.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-14T19:20:04.399 INFO:teuthology.orchestra.run.smithi082.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T19:20:04.842 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stdout 219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705 2023-07-14T19:20:04.842 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec... 2023-07-14T19:20:04.842 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Getting image source signatures 2023-07-14T19:20:04.842 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:0e454b422ba90e2ad13e636c40610401086a3576fe973b0460943839fe5f2508 2023-07-14T19:20:04.843 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-14T19:20:04.843 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Copying config sha256:219b180907a819a02c0d1790ec69e152120a3974adf061aaf947f90f6ccf9705 2023-07-14T19:20:04.843 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-14T19:20:04.843 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/podman: stderr Storing signatures 2023-07-14T19:20:05.690 INFO:teuthology.orchestra.run.smithi082.stdout:ceph: stdout ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T19:20:05.691 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph version: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T19:20:05.691 INFO:teuthology.orchestra.run.smithi082.stdout:Extracting ceph user uid/gid from container image... 2023-07-14T19:20:06.333 INFO:teuthology.orchestra.run.smithi082.stdout:stat: stdout 167 167 2023-07-14T19:20:06.333 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial keys... 2023-07-14T19:20:08.894 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQDnn7Fkuv/pMhAA+fKQsp+9DtpWIBv/hYUF7w== 2023-07-14T19:20:09.653 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQDpn7Fkc8R4ERAAwcBIARhTEr8FwsPbEChILA== 2023-07-14T19:20:10.308 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph-authtool: stdout AQDpn7Fkl8aXOhAAg8aIU15c+Xgz7CZqP+QG7g== 2023-07-14T19:20:10.309 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial monmap... 2023-07-14T19:20:11.062 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool for a [v2:172.21.15.82:3300,v1:172.21.15.82:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = pacific 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: set fsid to 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:11.063 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-14T19:20:11.064 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:20:11.064 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mon... 2023-07-14T19:20:11.811 INFO:teuthology.orchestra.run.smithi082.stdout:create mon.a on 2023-07-14T19:20:11.929 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-14T19:20:12.017 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-14T19:20:12.130 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c.target. 2023-07-14T19:20:12.131 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c.target → /etc/systemd/system/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c.target. 2023-07-14T19:20:12.301 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a 2023-07-14T19:20:12.301 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a.service: Unit ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a.service not loaded. 2023-07-14T19:20:12.387 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c.target.wants/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@.service. 2023-07-14T19:20:12.691 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:12 smithi082 systemd[1]: Starting Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:20:12.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:12 smithi082 podman[126909]: 2023-07-14T19:20:12.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:12 smithi082 bash[126909]: b99bfa52e4030b6de7ed6def057137c88583f435f0f1d3b720b11fa40de1d3e3 2023-07-14T19:20:12.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:12 smithi082 ceph-mon[126949]: mkfs 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:12.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:12 smithi082 ceph-mon[126949]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T19:20:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T19:20:12.989 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-07-14T19:20:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T19:20:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-07-14T19:20:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-07-14T19:20:12.996 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T19:20:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon to start... 2023-07-14T19:20:12.997 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mon... 2023-07-14T19:20:13.242 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:12 smithi082 systemd[1]: Started Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:20:13.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout cluster: 2023-07-14T19:20:13.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout id: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:13.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-14T19:20:13.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:13.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout services: 2023-07-14T19:20:13.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.616489s) 2023-07-14T19:20:13.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-14T19:20:13.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-14T19:20:13.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout data: 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout pgs: 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.453+0000 7f97d0012700 1 Processor -- start 2023-07-14T19:20:13.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.454+0000 7f97d0012700 1 -- start start 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.454+0000 7f97d0012700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 0x7f97c8107f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.454+0000 7f97d0012700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c81084d0 con 0x7f97c8079ce0 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.454+0000 7f97cddae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 0x7f97c8107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.454+0000 7f97cddae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 0x7f97c8107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39400/0 (socket says 172.21.15.82:39400) 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.454+0000 7f97cddae700 1 -- 172.21.15.82:0/1292255396 learned_addr learned my addr 172.21.15.82:0/1292255396 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97cddae700 1 -- 172.21.15.82:0/1292255396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c8108610 con 0x7f97c8079ce0 2023-07-14T19:20:13.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97cddae700 1 --2- 172.21.15.82:0/1292255396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 0x7f97c8107f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f97b80098a0 tx=0x7f97b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23daa0773d87539 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97ccdac700 1 -- 172.21.15.82:0/1292255396 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97b8004160 con 0x7f97c8079ce0 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97ccdac700 1 -- 172.21.15.82:0/1292255396 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b80042c0 con 0x7f97c8079ce0 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97ccdac700 1 -- 172.21.15.82:0/1292255396 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97b80045a0 con 0x7f97c8079ce0 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97d0012700 1 -- 172.21.15.82:0/1292255396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 msgr2=0x7f97c8107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.455+0000 7f97d0012700 1 --2- 172.21.15.82:0/1292255396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 0x7f97c8107f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f97b80098a0 tx=0x7f97b8009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 -- 172.21.15.82:0/1292255396 shutdown_connections 2023-07-14T19:20:13.850 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 --2- 172.21.15.82:0/1292255396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c8079ce0 0x7f97c8107f90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 -- 172.21.15.82:0/1292255396 >> 172.21.15.82:0/1292255396 conn(0x7f97c80ff9a0 msgr2=0x7f97c8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 -- 172.21.15.82:0/1292255396 shutdown_connections 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 -- 172.21.15.82:0/1292255396 wait complete. 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 Processor -- start 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 -- start start 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 0x7f97c8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.456+0000 7f97d0012700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8199230 con 0x7f97c81988e0 2023-07-14T19:20:13.851 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97cddae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 0x7f97c8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97cddae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 0x7f97c8198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39402/0 (socket says 172.21.15.82:39402) 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97cddae700 1 -- 172.21.15.82:0/64383850 learned_addr learned my addr 172.21.15.82:0/64383850 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97cddae700 1 -- 172.21.15.82:0/64383850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009550 con 0x7f97c81988e0 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97cddae700 1 --2- 172.21.15.82:0/64383850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 0x7f97c8198cf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004000 tx=0x7f97b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97b8011630 con 0x7f97c81988e0 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8011790 con 0x7f97c81988e0 2023-07-14T19:20:13.852 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c8199430 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97b8011a30 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.457+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c819c090 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.458+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f97b8022070 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.458+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f97b801c450 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.458+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c81926e0 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.460+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f97b801b450 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f97c8079ce0 con 0x7f97c81988e0 2023-07-14T19:20:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97beffd700 1 -- 172.21.15.82:0/64383850 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f97b801b5f0 con 0x7f97c81988e0 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 msgr2=0x7f97c8198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97d0012700 1 --2- 172.21.15.82:0/64383850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 0x7f97c8198cf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004000 tx=0x7f97b8004750 comp rx=0 tx=0).stop 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 shutdown_connections 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97d0012700 1 --2- 172.21.15.82:0/64383850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c81988e0 0x7f97c8198cf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.530+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 >> 172.21.15.82:0/64383850 conn(0x7f97c80ff9a0 msgr2=0x7f97c8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.531+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 shutdown_connections 2023-07-14T19:20:13.854 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:13.531+0000 7f97d0012700 1 -- 172.21.15.82:0/64383850 wait complete. 2023-07-14T19:20:13.855 INFO:teuthology.orchestra.run.smithi082.stdout:mon is available 2023-07-14T19:20:13.855 INFO:teuthology.orchestra.run.smithi082.stdout:Assimilating anything we can from ceph.conf... 2023-07-14T19:20:14.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:13 smithi082 ceph-mon[126949]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T19:20:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:13 smithi082 ceph-mon[126949]: monmap e1: 1 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-07-14T19:20:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:13 smithi082 ceph-mon[126949]: fsmap 2023-07-14T19:20:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:13 smithi082 ceph-mon[126949]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T19:20:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:13 smithi082 ceph-mon[126949]: mgrmap e1: no daemons active 2023-07-14T19:20:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:13 smithi082 ceph-mon[126949]: from='client.? 172.21.15.82:0/64383850' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-14T19:20:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T19:20:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.82:3300,v1:172.21.15.82:6789] 2023-07-14T19:20:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T19:20:14.789 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T19:20:14.799 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T19:20:14.800 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T19:20:14.800 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T19:20:14.800 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T19:20:14.800 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T19:20:14.800 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca8121b700 1 Processor -- start 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca8121b700 1 -- start start 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca8121b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca8121b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c079d00 con 0x7fca7c105f60 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca7ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca7ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39404/0 (socket says 172.21.15.82:39404) 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca7ad9d700 1 -- 172.21.15.82:0/2103587272 learned_addr learned my addr 172.21.15.82:0/2103587272 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca7ad9d700 1 -- 172.21.15.82:0/2103587272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c079e40 con 0x7fca7c105f60 2023-07-14T19:20:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.293+0000 7fca7ad9d700 1 --2- 172.21.15.82:0/2103587272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c108340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0098a0 tx=0x7fca6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9516743195f54f47 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca79d9b700 1 -- 172.21.15.82:0/2103587272 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca6c004160 con 0x7fca7c105f60 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca79d9b700 1 -- 172.21.15.82:0/2103587272 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c0042c0 con 0x7fca7c105f60 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca79d9b700 1 -- 172.21.15.82:0/2103587272 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca6c0045a0 con 0x7fca7c105f60 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 -- 172.21.15.82:0/2103587272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 msgr2=0x7fca7c108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 --2- 172.21.15.82:0/2103587272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c108340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0098a0 tx=0x7fca6c009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 -- 172.21.15.82:0/2103587272 shutdown_connections 2023-07-14T19:20:14.802 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 --2- 172.21.15.82:0/2103587272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c108340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 -- 172.21.15.82:0/2103587272 >> 172.21.15.82:0/2103587272 conn(0x7fca7c0ff9a0 msgr2=0x7fca7c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 -- 172.21.15.82:0/2103587272 shutdown_connections 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.294+0000 7fca8121b700 1 -- 172.21.15.82:0/2103587272 wait complete. 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca8121b700 1 Processor -- start 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca8121b700 1 -- start start 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca8121b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:14.803 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca8121b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c198e30 con 0x7fca7c105f60 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca7ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca7ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39406/0 (socket says 172.21.15.82:39406) 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.295+0000 7fca7ad9d700 1 -- 172.21.15.82:0/2390734401 learned_addr learned my addr 172.21.15.82:0/2390734401 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca7ad9d700 1 -- 172.21.15.82:0/2390734401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c009550 con 0x7fca7c105f60 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca7ad9d700 1 --2- 172.21.15.82:0/2390734401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c1988f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fca6c004000 tx=0x7fca6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca6c011630 con 0x7fca7c105f60 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c011790 con 0x7fca7c105f60 2023-07-14T19:20:14.804 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c199030 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca6c011a30 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c199450 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fca6c022070 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.296+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fca6c01c450 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.297+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c1926d0 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.299+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fca6c01b450 con 0x7fca7c105f60 2023-07-14T19:20:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.365+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fca7c061c70 con 0x7fca7c105f60 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.369+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca6c01b5f0 con 0x7fca7c105f60 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.369+0000 7fca63fff700 1 -- 172.21.15.82:0/2390734401 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fca6c024630 con 0x7fca7c105f60 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 msgr2=0x7fca7c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 --2- 172.21.15.82:0/2390734401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c1988f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fca6c004000 tx=0x7fca6c004750 comp rx=0 tx=0).stop 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 shutdown_connections 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 --2- 172.21.15.82:0/2390734401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7c105f60 0x7fca7c1988f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 >> 172.21.15.82:0/2390734401 conn(0x7fca7c0ff9a0 msgr2=0x7fca7c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:14.806 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 shutdown_connections 2023-07-14T19:20:14.807 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:14.370+0000 7fca8121b700 1 -- 172.21.15.82:0/2390734401 wait complete. 2023-07-14T19:20:14.807 INFO:teuthology.orchestra.run.smithi082.stdout:Generating new minimal ceph.conf... 2023-07-14T19:20:14.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:14 smithi082 ceph-mon[126949]: from='client.? 172.21.15.82:0/2390734401' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T19:20:14.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:14 smithi082 ceph-mon[126949]: from='client.? 172.21.15.82:0/2390734401' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-14T19:20:15.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f942592f700 1 Processor -- start 2023-07-14T19:20:15.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f942592f700 1 -- start start 2023-07-14T19:20:15.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f942592f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420103af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:15.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f942592f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420104030 con 0x7f94201036e0 2023-07-14T19:20:15.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f941effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:15.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f941effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39408/0 (socket says 172.21.15.82:39408) 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.265+0000 7f941effd700 1 -- 172.21.15.82:0/3759088341 learned_addr learned my addr 172.21.15.82:0/3759088341 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f941effd700 1 -- 172.21.15.82:0/3759088341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9420104170 con 0x7f94201036e0 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f941effd700 1 --2- 172.21.15.82:0/3759088341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420103af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f94140098a0 tx=0x7f9414009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2390164c83103f64 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f941dffb700 1 -- 172.21.15.82:0/3759088341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9414004160 con 0x7f94201036e0 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f941dffb700 1 -- 172.21.15.82:0/3759088341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94140042c0 con 0x7f94201036e0 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f941dffb700 1 -- 172.21.15.82:0/3759088341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9414011a00 con 0x7f94201036e0 2023-07-14T19:20:15.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f942592f700 1 -- 172.21.15.82:0/3759088341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 msgr2=0x7f9420103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f942592f700 1 --2- 172.21.15.82:0/3759088341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420103af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f94140098a0 tx=0x7f9414009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f942592f700 1 -- 172.21.15.82:0/3759088341 shutdown_connections 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f942592f700 1 --2- 172.21.15.82:0/3759088341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420103af0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f942592f700 1 -- 172.21.15.82:0/3759088341 >> 172.21.15.82:0/3759088341 conn(0x7f94200ff120 msgr2=0x7f9420101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.266+0000 7f942592f700 1 -- 172.21.15.82:0/3759088341 shutdown_connections 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f942592f700 1 -- 172.21.15.82:0/3759088341 wait complete. 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f942592f700 1 Processor -- start 2023-07-14T19:20:15.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f942592f700 1 -- start start 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f942592f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f942592f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420199650 con 0x7f94201036e0 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f941effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f941effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39410/0 (socket says 172.21.15.82:39410) 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.267+0000 7f941effd700 1 -- 172.21.15.82:0/2133812911 learned_addr learned my addr 172.21.15.82:0/2133812911 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f941effd700 1 -- 172.21.15.82:0/2133812911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9414009550 con 0x7f94201036e0 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f941effd700 1 --2- 172.21.15.82:0/2133812911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420199110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9414009fb0 tx=0x7f9414011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:15.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9414003930 con 0x7f94201036e0 2023-07-14T19:20:15.638 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9420199850 con 0x7f94201036e0 2023-07-14T19:20:15.638 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9414003a90 con 0x7f94201036e0 2023-07-14T19:20:15.638 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94140195e0 con 0x7f94201036e0 2023-07-14T19:20:15.638 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9420199c70 con 0x7f94201036e0 2023-07-14T19:20:15.638 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9414019ba0 con 0x7f94201036e0 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.268+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f94140243f0 con 0x7f94201036e0 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.269+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420061c70 con 0x7f94201036e0 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.271+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f9414003c00 con 0x7f94201036e0 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.338+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9420107480 con 0x7f94201036e0 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.339+0000 7f942492d700 1 -- 172.21.15.82:0/2133812911 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f9420107480 con 0x7f94201036e0 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.339+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 msgr2=0x7f9420199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:15.639 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.339+0000 7f942592f700 1 --2- 172.21.15.82:0/2133812911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420199110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9414009fb0 tx=0x7f9414011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:15.640 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.340+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 shutdown_connections 2023-07-14T19:20:15.640 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.340+0000 7f942592f700 1 --2- 172.21.15.82:0/2133812911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94201036e0 0x7f9420199110 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:15.640 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.340+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 >> 172.21.15.82:0/2133812911 conn(0x7f94200ff120 msgr2=0x7f94200ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:15.640 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.340+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 shutdown_connections 2023-07-14T19:20:15.640 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:15.340+0000 7f942592f700 1 -- 172.21.15.82:0/2133812911 wait complete. 2023-07-14T19:20:15.640 INFO:teuthology.orchestra.run.smithi082.stdout:Restarting the monitor... 2023-07-14T19:20:15.923 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:15 smithi082 systemd[1]: Stopping Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:20:16.186 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:15 smithi082 ceph-mon[126949]: from='client.? 172.21.15.82:0/2133812911' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:20:16.186 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:15 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[126925]: 2023-07-14T19:20:15.967+0000 7f9290786700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:20:16.186 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:15 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[126925]: 2023-07-14T19:20:15.967+0000 7f9290786700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-14T19:20:16.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 bash[127293]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a 2023-07-14T19:20:16.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a.service: Succeeded. 2023-07-14T19:20:16.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 systemd[1]: Stopped Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:20:16.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 systemd[1]: Starting Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:20:17.038 INFO:teuthology.orchestra.run.smithi082.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-14T19:20:17.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 podman[127421]: 2023-07-14T19:20:17.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 ceph-mon[127461]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 ceph-mon[127461]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev), process ceph-mon, pid 7 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:16 smithi082 ceph-mon[127461]: pidfile_write: ignore empty --pid-file 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 bash[127421]: 8c88ed0430c20dd465d560e1bd09236b76aa181c0616d53964a53818af0f5bd4 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: load: jerasure load: lrc 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: RocksDB version: 7.9.2 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Git sha 0 2023-07-14T19:20:17.218 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Compile date 2023-07-14 14:45:42 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: DB SUMMARY 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: DB Session ID: 3DHZRWJTJYFYSU9ZNSHX 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: CURRENT file: CURRENT 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-14T19:20:17.219 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98685 ; 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.error_if_exists: 0 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.create_if_missing: 0 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.paranoid_checks: 1 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T19:20:17.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.env: 0x562e4dc65c60 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.info_log: 0x562e4e93f2a0 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.statistics: (nil) 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.use_fsync: 0 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_log_file_size: 0 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T19:20:17.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T19:20:17.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T19:20:17.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T19:20:17.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.allow_fallocate: 1 2023-07-14T19:20:17.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T19:20:17.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T19:20:17.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.use_direct_reads: 0 2023-07-14T19:20:17.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T19:20:17.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T19:20:17.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.db_log_dir: 2023-07-14T19:20:17.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.wal_dir: 2023-07-14T19:20:17.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T19:20:17.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T19:20:17.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T19:20:17.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T19:20:17.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.write_buffer_manager: 0x562e4e8603c0 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T19:20:17.226 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T19:20:17.227 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T19:20:17.227 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T19:20:17.227 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T19:20:17.227 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T19:20:17.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T19:20:17.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.unordered_write: 0 2023-07-14T19:20:17.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T19:20:17.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T19:20:17.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T19:20:17.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T19:20:17.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.row_cache: None 2023-07-14T19:20:17.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.wal_filter: None 2023-07-14T19:20:17.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T19:20:17.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T19:20:17.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.two_write_queues: 0 2023-07-14T19:20:17.230 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T19:20:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.wal_compression: 0 2023-07-14T19:20:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.atomic_flush: 0 2023-07-14T19:20:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T19:20:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T19:20:17.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T19:20:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.log_readahead_size: 0 2023-07-14T19:20:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T19:20:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T19:20:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T19:20:17.232 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T19:20:17.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T19:20:17.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T19:20:17.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T19:20:17.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_background_jobs: 2 2023-07-14T19:20:17.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_background_compactions: -1 2023-07-14T19:20:17.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_subcompactions: 1 2023-07-14T19:20:17.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T19:20:17.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T19:20:17.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_open_files: -1 2023-07-14T19:20:17.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T19:20:17.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T19:20:17.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T19:20:17.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T19:20:17.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_background_flushes: -1 2023-07-14T19:20:17.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Compression algorithms supported: 2023-07-14T19:20:17.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T19:20:17.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kZSTD supported: 0 2023-07-14T19:20:17.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kXpressCompression supported: 0 2023-07-14T19:20:17.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T19:20:17.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kZlibCompression supported: 1 2023-07-14T19:20:17.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kSnappyCompression supported: 1 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kLZ4Compression supported: 1 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: kBZip2Compression supported: 0 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T19:20:17.239 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T19:20:17.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.merge_operator: 2023-07-14T19:20:17.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_filter: None 2023-07-14T19:20:17.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e4e93f3a0) 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks: 1 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T19:20:17.241 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T19:20:17.242 INFO:journalctl@ceph.mon.a.smithi082.stdout: pin_top_level_index_and_filter: 1 2023-07-14T19:20:17.242 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_type: 0 2023-07-14T19:20:17.242 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_index_type: 0 2023-07-14T19:20:17.242 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_shortening: 1 2023-07-14T19:20:17.242 INFO:journalctl@ceph.mon.a.smithi082.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T19:20:17.242 INFO:journalctl@ceph.mon.a.smithi082.stdout: checksum: 4 2023-07-14T19:20:17.243 INFO:journalctl@ceph.mon.a.smithi082.stdout: no_block_cache: 0 2023-07-14T19:20:17.243 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache: 0x562e4e8ff1f0 2023-07-14T19:20:17.243 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_name: BinnedLRUCache 2023-07-14T19:20:17.243 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_options: 2023-07-14T19:20:17.243 INFO:journalctl@ceph.mon.a.smithi082.stdout: capacity : 536870912 2023-07-14T19:20:17.243 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_shard_bits : 4 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: strict_capacity_limit : 0 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: high_pri_pool_ratio: 0.000 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_cache_compressed: (nil) 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: persistent_cache: (nil) 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size: 4096 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_size_deviation: 10 2023-07-14T19:20:17.244 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_restart_interval: 16 2023-07-14T19:20:17.245 INFO:journalctl@ceph.mon.a.smithi082.stdout: index_block_restart_interval: 1 2023-07-14T19:20:17.245 INFO:journalctl@ceph.mon.a.smithi082.stdout: metadata_block_size: 4096 2023-07-14T19:20:17.245 INFO:journalctl@ceph.mon.a.smithi082.stdout: partition_filters: 0 2023-07-14T19:20:17.245 INFO:journalctl@ceph.mon.a.smithi082.stdout: use_delta_encoding: 1 2023-07-14T19:20:17.245 INFO:journalctl@ceph.mon.a.smithi082.stdout: filter_policy: bloomfilter 2023-07-14T19:20:17.246 INFO:journalctl@ceph.mon.a.smithi082.stdout: whole_key_filtering: 1 2023-07-14T19:20:17.246 INFO:journalctl@ceph.mon.a.smithi082.stdout: verify_compression: 0 2023-07-14T19:20:17.246 INFO:journalctl@ceph.mon.a.smithi082.stdout: read_amp_bytes_per_bit: 0 2023-07-14T19:20:17.246 INFO:journalctl@ceph.mon.a.smithi082.stdout: format_version: 5 2023-07-14T19:20:17.247 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_index_compression: 1 2023-07-14T19:20:17.247 INFO:journalctl@ceph.mon.a.smithi082.stdout: block_align: 0 2023-07-14T19:20:17.247 INFO:journalctl@ceph.mon.a.smithi082.stdout: max_auto_readahead_size: 262144 2023-07-14T19:20:17.247 INFO:journalctl@ceph.mon.a.smithi082.stdout: prepopulate_block_cache: 0 2023-07-14T19:20:17.248 INFO:journalctl@ceph.mon.a.smithi082.stdout: initial_auto_readahead_size: 8192 2023-07-14T19:20:17.248 INFO:journalctl@ceph.mon.a.smithi082.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T19:20:17.248 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T19:20:17.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T19:20:17.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression: NoCompression 2023-07-14T19:20:17.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T19:20:17.249 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T19:20:17.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T19:20:17.250 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.num_levels: 7 2023-07-14T19:20:17.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T19:20:17.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T19:20:17.251 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T19:20:17.252 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T19:20:17.252 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T19:20:17.252 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T19:20:17.252 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T19:20:17.253 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T19:20:17.253 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T19:20:17.253 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T19:20:17.253 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T19:20:17.253 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T19:20:17.254 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T19:20:17.254 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T19:20:17.254 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T19:20:17.254 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T19:20:17.254 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T19:20:17.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T19:20:17.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T19:20:17.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T19:20:17.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T19:20:17.255 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T19:20:17.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T19:20:17.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T19:20:17.256 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T19:20:17.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T19:20:17.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T19:20:17.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T19:20:17.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T19:20:17.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T19:20:17.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T19:20:17.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T19:20:17.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T19:20:17.259 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T19:20:17.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T19:20:17.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T19:20:17.260 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T19:20:17.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T19:20:17.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T19:20:17.261 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T19:20:17.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T19:20:17.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T19:20:17.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T19:20:17.262 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T19:20:17.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T19:20:17.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T19:20:17.263 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T19:20:17.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T19:20:17.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T19:20:17.264 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T19:20:17.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T19:20:17.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T19:20:17.265 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T19:20:17.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.table_properties_collectors: 2023-07-14T19:20:17.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.inplace_update_support: 0 2023-07-14T19:20:17.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T19:20:17.266 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T19:20:17.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T19:20:17.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T19:20:17.267 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.bloom_locality: 0 2023-07-14T19:20:17.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.max_successive_merges: 0 2023-07-14T19:20:17.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T19:20:17.268 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T19:20:17.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T19:20:17.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T19:20:17.269 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.ttl: 2592000 2023-07-14T19:20:17.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T19:20:17.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T19:20:17.270 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T19:20:17.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.enable_blob_files: false 2023-07-14T19:20:17.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.min_blob_size: 0 2023-07-14T19:20:17.271 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T19:20:17.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T19:20:17.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T19:20:17.272 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T19:20:17.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T19:20:17.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T19:20:17.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T19:20:17.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T19:20:17.273 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-14T19:20:17.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-14T19:20:17.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 86b343e8-6dc3-4c5e-9674-8eff27f08c49 2023-07-14T19:20:17.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362417005443, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-14T19:20:17.274 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-14T19:20:17.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362417006375, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91931, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13640, "raw_average_key_size": 49, "raw_value_size": 84496, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689362417, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "86b343e8-6dc3-4c5e-9674-8eff27f08c49", "db_session_id": "3DHZRWJTJYFYSU9ZNSHX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-14T19:20:17.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362417006450, "job": 1, "event": "recovery_finished"} 2023-07-14T19:20:17.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-14T19:20:17.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T19:20:17.275 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e4e998000 2023-07-14T19:20:17.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: DB pointer 0x562e4e984000 2023-07-14T19:20:17.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T19:20:17.276 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T19:20:17.276 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** DB Stats ** 2023-07-14T19:20:17.276 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T19:20:17.276 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-07-14T19:20:17.277 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: L0 2/0 93.56 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: Sum 2/0 93.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-07-14T19:20:17.278 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** Compaction Stats [default] ** 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 161.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-07-14T19:20:17.279 INFO:journalctl@ceph.mon.a.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T19:20:17.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T19:20:17.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T19:20:17.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T19:20:17.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T19:20:17.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T19:20:17.280 INFO:journalctl@ceph.mon.a.smithi082.stdout: Cumulative compaction: 0.00 GB write, 28.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T19:20:17.281 INFO:journalctl@ceph.mon.a.smithi082.stdout: Interval compaction: 0.00 GB write, 28.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T19:20:17.281 INFO:journalctl@ceph.mon.a.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T19:20:17.281 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache BinnedLRUCache@0x562e4e8ff1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-14T19:20:17.281 INFO:journalctl@ceph.mon.a.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-14T19:20:17.281 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: starting mon.a rank 0 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???) e1 preinit fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).mds e1 new map 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).mds e1 print_map 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: e1 2023-07-14T19:20:17.282 INFO:journalctl@ceph.mon.a.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: legacy client fscid: -1 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout: No filesystems configured 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:20:17.283 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:20:17.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:20:17.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-14T19:20:17.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 systemd[1]: Started Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:20:17.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-14T19:20:17.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: monmap e1: 1 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-07-14T19:20:17.284 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: fsmap 2023-07-14T19:20:17.285 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: osdmap e1: 0 total, 0 up, 0 in 2023-07-14T19:20:17.285 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:17 smithi082 ceph-mon[127461]: mgrmap e1: no daemons active 2023-07-14T19:20:17.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.518+0000 7fa6491fb700 1 Processor -- start 2023-07-14T19:20:17.990 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.518+0000 7fa6491fb700 1 -- start start 2023-07-14T19:20:17.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa6491fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:17.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa6491fb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa644104c40 con 0x7fa6441042f0 2023-07-14T19:20:17.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa642d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:17.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa642d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39412/0 (socket says 172.21.15.82:39412) 2023-07-14T19:20:17.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa642d9d700 1 -- 172.21.15.82:0/1652677173 learned_addr learned my addr 172.21.15.82:0/1652677173 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:17.991 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa642d9d700 1 -- 172.21.15.82:0/1652677173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644104d80 con 0x7fa6441042f0 2023-07-14T19:20:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.519+0000 7fa642d9d700 1 --2- 172.21.15.82:0/1652677173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644104700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098a0 tx=0x7fa638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bf7ad2a336e8d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa641d9b700 1 -- 172.21.15.82:0/1652677173 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa638004160 con 0x7fa6441042f0 2023-07-14T19:20:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa641d9b700 1 -- 172.21.15.82:0/1652677173 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa6380042c0 con 0x7fa6441042f0 2023-07-14T19:20:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa641d9b700 1 -- 172.21.15.82:0/1652677173 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa638011a00 con 0x7fa6441042f0 2023-07-14T19:20:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 -- 172.21.15.82:0/1652677173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 msgr2=0x7fa644104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:17.992 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 --2- 172.21.15.82:0/1652677173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644104700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6380098a0 tx=0x7fa638009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 -- 172.21.15.82:0/1652677173 shutdown_connections 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 --2- 172.21.15.82:0/1652677173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644104700 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 -- 172.21.15.82:0/1652677173 >> 172.21.15.82:0/1652677173 conn(0x7fa6440ff9a0 msgr2=0x7fa644101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 -- 172.21.15.82:0/1652677173 shutdown_connections 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.520+0000 7fa6491fb700 1 -- 172.21.15.82:0/1652677173 wait complete. 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa6491fb700 1 Processor -- start 2023-07-14T19:20:17.993 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa6491fb700 1 -- start start 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa6491fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644079b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa6491fb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64419bdf0 con 0x7fa6441042f0 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa642d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644079b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa642d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644079b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39414/0 (socket says 172.21.15.82:39414) 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.521+0000 7fa642d9d700 1 -- 172.21.15.82:0/548488236 learned_addr learned my addr 172.21.15.82:0/548488236 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa642d9d700 1 -- 172.21.15.82:0/548488236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638009550 con 0x7fa6441042f0 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa642d9d700 1 --2- 172.21.15.82:0/548488236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644079b30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa638009fb0 tx=0x7fa638011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:17.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa638003930 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64419bf30 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa638003a90 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa638019610 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.522+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64419c350 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.523+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa638019bd0 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.523+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa63801a430 con 0x7fa6441042f0 2023-07-14T19:20:17.995 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.523+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa644061c70 con 0x7fa6441042f0 2023-07-14T19:20:17.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.525+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa638003c00 con 0x7fa6441042f0 2023-07-14T19:20:17.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.592+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa6441061b0 con 0x7fa6441042f0 2023-07-14T19:20:17.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.594+0000 7fa62bfff700 1 -- 172.21.15.82:0/548488236 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa63801a9d0 con 0x7fa6441042f0 2023-07-14T19:20:17.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 msgr2=0x7fa644079b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:17.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 --2- 172.21.15.82:0/548488236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644079b30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa638009fb0 tx=0x7fa638011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:17.996 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 shutdown_connections 2023-07-14T19:20:17.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 --2- 172.21.15.82:0/548488236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6441042f0 0x7fa644079b30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:17.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 >> 172.21.15.82:0/548488236 conn(0x7fa6440ff9a0 msgr2=0x7fa644100740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:17.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 shutdown_connections 2023-07-14T19:20:17.997 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:17.595+0000 7fa6491fb700 1 -- 172.21.15.82:0/548488236 wait complete. 2023-07-14T19:20:17.997 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-14T19:20:18.002 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:20:18.003 INFO:teuthology.orchestra.run.smithi082.stdout:Creating mgr... 2023-07-14T19:20:18.003 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 9283 ... 2023-07-14T19:20:18.003 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying port 8765 ... 2023-07-14T19:20:18.197 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T19:20:18.197 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x.service: Unit ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x.service not loaded. 2023-07-14T19:20:18.285 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c.target.wants/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@.service. 2023-07-14T19:20:18.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:18 smithi082 systemd[1]: Starting Ceph mgr.x for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:20:18.876 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:18 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/548488236' entity='client.admin' 2023-07-14T19:20:18.876 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:18 smithi082 podman[127690]: 2023-07-14T19:20:18.877 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:18 smithi082 bash[127690]: 85707f90267572ff80368756635c7c5b47a8a4f5b198f9bafd787ae764e007a0 2023-07-14T19:20:18.888 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T19:20:18.892 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-07-14T19:20:18.896 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T19:20:18.897 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-07-14T19:20:18.897 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-07-14T19:20:18.897 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to enable service . firewalld.service is not available 2023-07-14T19:20:18.904 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T19:20:18.904 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-07-14T19:20:18.911 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T19:20:18.912 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-07-14T19:20:18.912 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-07-14T19:20:18.912 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-14T19:20:18.912 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr to start... 2023-07-14T19:20:18.912 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr... 2023-07-14T19:20:19.150 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:18 smithi082 systemd[1]: Started Ceph mgr.x for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:20:19.151 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:18 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:18.936+0000 7f194945a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:20:19.151 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.018+0000 7f194945a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:20:19.151 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.108+0000 7f194945a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:20:19.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.210+0000 7f194945a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:20:19.465 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.387+0000 7f194945a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:20:19.815 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2061095746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T19:20:19.816 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.724+0000 7f194945a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:20:19.817 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.813+0000 7f194945a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:20:19.859 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:19.859 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:19.860 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:20:19.860 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T19:20:19.860 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T19:20:19.860 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T19:20:19.861 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T19:20:19.861 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.861 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T19:20:19.862 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T19:20:19.862 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-07-14T19:20:19.862 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:19.862 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T19:20:19.862 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T19:20:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-07-14T19:20:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T19:20:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T19:20:19.864 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T19:20:19.870 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T19:20:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T19:20:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T19:20:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T19:20:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T19:20:19.871 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.872 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T19:20:19.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T19:20:12.916427+0000", 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64dd9df700 1 Processor -- start 2023-07-14T19:20:19.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64dd9df700 1 -- start start 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64dd9df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64dd9df700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8079d20 con 0x7f64d8105f40 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39420/0 (socket says 172.21.15.82:39420) 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.394+0000 7f64d6ffd700 1 -- 172.21.15.82:0/1944504582 learned_addr learned my addr 172.21.15.82:0/1944504582 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64d6ffd700 1 -- 172.21.15.82:0/1944504582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d8079e60 con 0x7f64d8105f40 2023-07-14T19:20:19.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64d6ffd700 1 --2- 172.21.15.82:0/1944504582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8108320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098a0 tx=0x7f64c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cb1b79b3c5c395 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64d5ffb700 1 -- 172.21.15.82:0/1944504582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64c8004160 con 0x7f64d8105f40 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64d5ffb700 1 -- 172.21.15.82:0/1944504582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64c80042c0 con 0x7f64d8105f40 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64d5ffb700 1 -- 172.21.15.82:0/1944504582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64c8011a00 con 0x7f64d8105f40 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64dd9df700 1 -- 172.21.15.82:0/1944504582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 msgr2=0x7f64d8108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.395+0000 7f64dd9df700 1 --2- 172.21.15.82:0/1944504582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8108320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098a0 tx=0x7f64c8009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.396+0000 7f64dd9df700 1 -- 172.21.15.82:0/1944504582 shutdown_connections 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.396+0000 7f64dd9df700 1 --2- 172.21.15.82:0/1944504582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8108320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:19.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.396+0000 7f64dd9df700 1 -- 172.21.15.82:0/1944504582 >> 172.21.15.82:0/1944504582 conn(0x7f64d80ff9a0 msgr2=0x7f64d8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.396+0000 7f64dd9df700 1 -- 172.21.15.82:0/1944504582 shutdown_connections 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.396+0000 7f64dd9df700 1 -- 172.21.15.82:0/1944504582 wait complete. 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.396+0000 7f64dd9df700 1 Processor -- start 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64dd9df700 1 -- start start 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64dd9df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64dd9df700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81995c0 con 0x7f64d8105f40 2023-07-14T19:20:19.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64d6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39422/0 (socket says 172.21.15.82:39422) 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64d6ffd700 1 -- 172.21.15.82:0/2061095746 learned_addr learned my addr 172.21.15.82:0/2061095746 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64d6ffd700 1 -- 172.21.15.82:0/2061095746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009550 con 0x7f64d8105f40 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.397+0000 7f64d6ffd700 1 --2- 172.21.15.82:0/2061095746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8199080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009fb0 tx=0x7f64c8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64c8003930 con 0x7f64d8105f40 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64c8003a90 con 0x7f64d8105f40 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d8199820 con 0x7f64d8105f40 2023-07-14T19:20:19.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64c80195e0 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d8199ca0 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f64c80198a0 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.398+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f64c802da10 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.399+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d8109460 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.401+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f64c8003c00 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.483+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f64d8109650 con 0x7f64d8105f40 2023-07-14T19:20:19.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.483+0000 7f64dc9dd700 1 -- 172.21.15.82:0/2061095746 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f64c802d440 con 0x7f64d8105f40 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 msgr2=0x7f64d8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 --2- 172.21.15.82:0/2061095746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8199080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009fb0 tx=0x7f64c8011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 shutdown_connections 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 --2- 172.21.15.82:0/2061095746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64d8105f40 0x7f64d8199080 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 >> 172.21.15.82:0/2061095746 conn(0x7f64d80ff9a0 msgr2=0x7f64d8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 shutdown_connections 2023-07-14T19:20:19.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:19.484+0000 7f64dd9df700 1 -- 172.21.15.82:0/2061095746 wait complete. 2023-07-14T19:20:19.882 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (1/15)... 2023-07-14T19:20:20.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:19 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:19.945+0000 7f194945a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:20:20.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:20 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:20.039+0000 7f194945a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:20:20.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:20 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:20.454+0000 7f194945a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:20:21.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:21 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:21.023+0000 7f194945a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:20:21.465 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:21 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:21.189+0000 7f194945a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:20:22.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:21 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:21.951+0000 7f194945a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:20:22.215 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:22.046+0000 7f194945a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:20:22.514 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:22.222+0000 7f194945a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:20:22.809 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:22.809 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T19:20:22.810 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-07-14T19:20:22.811 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T19:20:22.812 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T19:20:22.813 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T19:20:22.821 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T19:20:22.821 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T19:20:22.821 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T19:20:22.821 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.822 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T19:20:22.822 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T19:20:22.822 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T19:20:22.822 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T19:20:22.822 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T19:20:22.822 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T19:20:22.823 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T19:20:22.823 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T19:20:22.823 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T19:20:22.823 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.823 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T19:20:22.823 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T19:20:22.824 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.825 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T19:20:22.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:22.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T19:20:12.916427+0000", 2023-07-14T19:20:22.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:22.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:22.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T19:20:22.826 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb599f700 1 Processor -- start 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb599f700 1 -- start start 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb599f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 0x7fecb010a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb599f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0105bb0 con 0x7fecb0079ce0 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb499d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 0x7fecb010a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb499d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 0x7fecb010a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39424/0 (socket says 172.21.15.82:39424) 2023-07-14T19:20:22.827 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.327+0000 7fecb499d700 1 -- 172.21.15.82:0/2668624533 learned_addr learned my addr 172.21.15.82:0/2668624533 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.328+0000 7fecb499d700 1 -- 172.21.15.82:0/2668624533 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb010a700 con 0x7fecb0079ce0 2023-07-14T19:20:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.328+0000 7fecb499d700 1 --2- 172.21.15.82:0/2668624533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 0x7fecb010a1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feca40098a0 tx=0x7feca4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8781668362df430d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.328+0000 7fecaf7fe700 1 -- 172.21.15.82:0/2668624533 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feca4004160 con 0x7fecb0079ce0 2023-07-14T19:20:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.328+0000 7fecaf7fe700 1 -- 172.21.15.82:0/2668624533 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feca40042c0 con 0x7fecb0079ce0 2023-07-14T19:20:22.828 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.328+0000 7fecaf7fe700 1 -- 172.21.15.82:0/2668624533 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feca4011a00 con 0x7fecb0079ce0 2023-07-14T19:20:22.829 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 -- 172.21.15.82:0/2668624533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 msgr2=0x7fecb010a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:22.829 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 --2- 172.21.15.82:0/2668624533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 0x7fecb010a1c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feca40098a0 tx=0x7feca4009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:22.829 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 -- 172.21.15.82:0/2668624533 shutdown_connections 2023-07-14T19:20:22.829 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 --2- 172.21.15.82:0/2668624533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb0079ce0 0x7fecb010a1c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:22.829 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 -- 172.21.15.82:0/2668624533 >> 172.21.15.82:0/2668624533 conn(0x7fecb00ff770 msgr2=0x7fecb0101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 -- 172.21.15.82:0/2668624533 shutdown_connections 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 -- 172.21.15.82:0/2668624533 wait complete. 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 Processor -- start 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.329+0000 7fecb599f700 1 -- start start 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb599f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 0x7fecb019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb599f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb019baa0 con 0x7fecb019b150 2023-07-14T19:20:22.830 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb499d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 0x7fecb019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb499d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 0x7fecb019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39426/0 (socket says 172.21.15.82:39426) 2023-07-14T19:20:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb499d700 1 -- 172.21.15.82:0/3358287407 learned_addr learned my addr 172.21.15.82:0/3358287407 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb499d700 1 -- 172.21.15.82:0/3358287407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca4009550 con 0x7fecb019b150 2023-07-14T19:20:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb499d700 1 --2- 172.21.15.82:0/3358287407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 0x7fecb019b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feca4009fb0 tx=0x7feca4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feca4003a40 con 0x7fecb019b150 2023-07-14T19:20:22.831 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feca4003ba0 con 0x7fecb019b150 2023-07-14T19:20:22.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb019bca0 con 0x7fecb019b150 2023-07-14T19:20:22.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.330+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feca401a750 con 0x7fecb019b150 2023-07-14T19:20:22.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.331+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb019e900 con 0x7fecb019b150 2023-07-14T19:20:22.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.331+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feca401acd0 con 0x7fecb019b150 2023-07-14T19:20:22.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.331+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feca40193c0 con 0x7fecb019b150 2023-07-14T19:20:22.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.332+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb0061c70 con 0x7fecb019b150 2023-07-14T19:20:22.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.334+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7feca4024b60 con 0x7fecb019b150 2023-07-14T19:20:22.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.403+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fecb0079ce0 con 0x7fecb019b150 2023-07-14T19:20:22.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.404+0000 7fecadffb700 1 -- 172.21.15.82:0/3358287407 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7feca4024d00 con 0x7fecb019b150 2023-07-14T19:20:22.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.404+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 msgr2=0x7fecb019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:22.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.404+0000 7fecb599f700 1 --2- 172.21.15.82:0/3358287407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 0x7fecb019b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feca4009fb0 tx=0x7feca4011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:22.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.405+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 shutdown_connections 2023-07-14T19:20:22.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.405+0000 7fecb599f700 1 --2- 172.21.15.82:0/3358287407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecb019b150 0x7fecb019b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:22.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.405+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 >> 172.21.15.82:0/3358287407 conn(0x7fecb00ff770 msgr2=0x7fecb0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:22.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.405+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 shutdown_connections 2023-07-14T19:20:22.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:22.405+0000 7fecb599f700 1 -- 172.21.15.82:0/3358287407 wait complete. 2023-07-14T19:20:22.837 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (2/15)... 2023-07-14T19:20:22.893 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:22.511+0000 7f194945a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:20:22.893 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:22.742+0000 7f194945a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:20:23.153 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:23 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3358287407' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T19:20:23.154 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:22.891+0000 7f194945a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:20:23.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:23 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:23.151+0000 7f194945a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:20:24.160 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:23 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:23.841+0000 7f194945a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:20:24.160 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:23 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:23.970+0000 7f194945a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:20:24.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:24 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:24.158+0000 7f194945a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:20:24.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:24 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:24.270+0000 7f194945a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:20:24.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:24 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:24.485+0000 7f194945a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:20:25.622 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:25.340+0000 7f194945a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:20:25.623 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: Activating manager daemon x 2023-07-14T19:20:25.623 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: mgrmap e2: x(active, starting, since 0.00316475s) 2023-07-14T19:20:25.623 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T19:20:25.623 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T19:20:25.623 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: Manager daemon x is now available 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' 2023-07-14T19:20:25.624 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:25 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' 2023-07-14T19:20:26.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:26 smithi082 ceph-mon[127461]: from='mgr.14100 172.21.15.82:0/2501884820' entity='mgr.x' 2023-07-14T19:20:26.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1942083238' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T19:20:26.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:26 smithi082 ceph-mon[127461]: mgrmap e3: x(active, since 1.00624s) 2023-07-14T19:20:28.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:28.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:28.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:20:28.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T19:20:28.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T19:20:28.873 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:28.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T19:20:28.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T19:20:28.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:28.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T19:20:28.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T19:20:28.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T19:20:28.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T19:20:28.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T19:20:28.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T19:20:28.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T19:20:28.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T19:20:12.916427+0000", 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.068+0000 7ff4d107d700 1 Processor -- start 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.068+0000 7ff4d107d700 1 -- start start 2023-07-14T19:20:28.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.068+0000 7ff4d107d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.068+0000 7ff4d107d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc079d20 con 0x7ff4cc105f40 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.069+0000 7ff4cad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.069+0000 7ff4cad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39442/0 (socket says 172.21.15.82:39442) 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.069+0000 7ff4cad9d700 1 -- 172.21.15.82:0/3899494299 learned_addr learned my addr 172.21.15.82:0/3899494299 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.069+0000 7ff4cad9d700 1 -- 172.21.15.82:0/3899494299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc079e60 con 0x7ff4cc105f40 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.069+0000 7ff4cad9d700 1 --2- 172.21.15.82:0/3899494299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc108320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098a0 tx=0x7ff4bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d034ba8f2d1199 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4c9d9b700 1 -- 172.21.15.82:0/3899494299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc004160 con 0x7ff4cc105f40 2023-07-14T19:20:28.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4c9d9b700 1 -- 172.21.15.82:0/3899494299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4bc0042c0 con 0x7ff4cc105f40 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4c9d9b700 1 -- 172.21.15.82:0/3899494299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc011a00 con 0x7ff4cc105f40 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 -- 172.21.15.82:0/3899494299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 msgr2=0x7ff4cc108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 --2- 172.21.15.82:0/3899494299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc108320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098a0 tx=0x7ff4bc009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 -- 172.21.15.82:0/3899494299 shutdown_connections 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 --2- 172.21.15.82:0/3899494299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc108320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 -- 172.21.15.82:0/3899494299 >> 172.21.15.82:0/3899494299 conn(0x7ff4cc0ff9a0 msgr2=0x7ff4cc101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:28.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 -- 172.21.15.82:0/3899494299 shutdown_connections 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.070+0000 7ff4d107d700 1 -- 172.21.15.82:0/3899494299 wait complete. 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.071+0000 7ff4d107d700 1 Processor -- start 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4d107d700 1 -- start start 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4d107d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc1058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4d107d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc105de0 con 0x7ff4cc105f40 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4cad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc1058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:28.884 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4cad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc1058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39444/0 (socket says 172.21.15.82:39444) 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4cad9d700 1 -- 172.21.15.82:0/1942083238 learned_addr learned my addr 172.21.15.82:0/1942083238 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4cad9d700 1 -- 172.21.15.82:0/1942083238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009550 con 0x7ff4cc105f40 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4cad9d700 1 --2- 172.21.15.82:0/1942083238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc1058a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009fb0 tx=0x7ff4bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc003a40 con 0x7ff4cc105f40 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4bc003ba0 con 0x7ff4cc105f40 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.072+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc103f50 con 0x7ff4cc105f40 2023-07-14T19:20:28.885 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.073+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4bc01a750 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.073+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc1043d0 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.073+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 2) v1 ==== 45239+0+0 (secure 0 0 0) 0x7ff4bc01a980 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.074+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc061c70 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.074+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff4bc04bed0 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.076+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff4bc01ae10 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.150+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff4cc192b80 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.151+0000 7ff4c3fff700 1 -- 172.21.15.82:0/1942083238 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff4bc018350 con 0x7ff4cc105f40 2023-07-14T19:20:28.886 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.151+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 msgr2=0x7ff4cc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.151+0000 7ff4d107d700 1 --2- 172.21.15.82:0/1942083238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc1058a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009fb0 tx=0x7ff4bc011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.152+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 shutdown_connections 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.152+0000 7ff4d107d700 1 --2- 172.21.15.82:0/1942083238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc105f40 0x7ff4cc1058a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.152+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 >> 172.21.15.82:0/1942083238 conn(0x7ff4cc0ff9a0 msgr2=0x7ff4cc101100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.152+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 shutdown_connections 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:26.152+0000 7ff4d107d700 1 -- 172.21.15.82:0/1942083238 wait complete. 2023-07-14T19:20:28.887 INFO:teuthology.orchestra.run.smithi082.stdout:mgr not available, waiting (3/15)... 2023-07-14T19:20:29.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:29 smithi082 ceph-mon[127461]: mgrmap e4: x(active, since 2s) 2023-07-14T19:20:33.798 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:33 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3377097206' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-14T19:20:33.832 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:33.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:33.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:20:33.833 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "health": { 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 0 2023-07-14T19:20:33.834 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "a" 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:33.835 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-14T19:20:33.836 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-14T19:20:33.837 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-14T19:20:33.838 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-14T19:20:33.839 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "restful" 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ], 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-14T19:20:33.840 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "modified": "2023-07-14T19:20:12.916427+0000", 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout }, 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cc227f700 1 Processor -- start 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cc227f700 1 -- start start 2023-07-14T19:20:33.841 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cc227f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:33.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cc227f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc104c40 con 0x7f9cbc1042f0 2023-07-14T19:20:33.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:33.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39456/0 (socket says 172.21.15.82:39456) 2023-07-14T19:20:33.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.189+0000 7f9cbbfff700 1 -- 172.21.15.82:0/2509614326 learned_addr learned my addr 172.21.15.82:0/2509614326 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:33.842 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.190+0000 7f9cbbfff700 1 -- 172.21.15.82:0/2509614326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc104d80 con 0x7f9cbc1042f0 2023-07-14T19:20:33.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.190+0000 7f9cbbfff700 1 --2- 172.21.15.82:0/2509614326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc104700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0098a0 tx=0x7f9cac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6a8a550597e895 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:33.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.190+0000 7f9cbaffd700 1 -- 172.21.15.82:0/2509614326 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cac004160 con 0x7f9cbc1042f0 2023-07-14T19:20:33.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.190+0000 7f9cbaffd700 1 -- 172.21.15.82:0/2509614326 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9cac0042c0 con 0x7f9cbc1042f0 2023-07-14T19:20:33.843 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.190+0000 7f9cbaffd700 1 -- 172.21.15.82:0/2509614326 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cac011a00 con 0x7f9cbc1042f0 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 -- 172.21.15.82:0/2509614326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 msgr2=0x7f9cbc104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 --2- 172.21.15.82:0/2509614326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc104700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0098a0 tx=0x7f9cac009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 -- 172.21.15.82:0/2509614326 shutdown_connections 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 --2- 172.21.15.82:0/2509614326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc104700 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 -- 172.21.15.82:0/2509614326 >> 172.21.15.82:0/2509614326 conn(0x7f9cbc0ff9a0 msgr2=0x7f9cbc101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 -- 172.21.15.82:0/2509614326 shutdown_connections 2023-07-14T19:20:33.844 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 -- 172.21.15.82:0/2509614326 wait complete. 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 Processor -- start 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.191+0000 7f9cc227f700 1 -- start start 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cc227f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc079b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cc227f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc19be10 con 0x7f9cbc1042f0 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc079b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cbbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc079b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39458/0 (socket says 172.21.15.82:39458) 2023-07-14T19:20:33.845 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cbbfff700 1 -- 172.21.15.82:0/3377097206 learned_addr learned my addr 172.21.15.82:0/3377097206 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cbbfff700 1 -- 172.21.15.82:0/3377097206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac009550 con 0x7f9cbc1042f0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cbbfff700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc079b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009fb0 tx=0x7f9cac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cac003930 con 0x7f9cbc1042f0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9cac003a90 con 0x7f9cbc1042f0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc19bf50 con 0x7f9cbc1042f0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cac019610 con 0x7f9cbc1042f0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.192+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc19c310 con 0x7f9cbc1042f0 2023-07-14T19:20:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.193+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f9cac01a430 con 0x7f9cbc1042f0 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.193+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc061c70 con 0x7f9cbc1042f0 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.194+0000 7f9cb97fa700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f9ca8038670 0x7f9ca803ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.194+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9cac04c400 con 0x7f9cbc1042f0 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.194+0000 7f9cbb7fe700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f9ca8038670 0x7f9ca803ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.194+0000 7f9cbb7fe700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f9ca8038670 0x7f9ca803ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006fd0 tx=0x7f9ca4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.198+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9cac003c00 con 0x7f9cbc1042f0 2023-07-14T19:20:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.457+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9cbc1061b0 con 0x7f9cbc1042f0 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.457+0000 7f9cb97fa700 1 -- 172.21.15.82:0/3377097206 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9cac023b80 con 0x7f9cbc1042f0 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f9ca8038670 msgr2=0x7f9ca803ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f9ca8038670 0x7f9ca803ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006fd0 tx=0x7f9ca4006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 msgr2=0x7f9cbc079b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc079b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009fb0 tx=0x7f9cac011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 shutdown_connections 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f9ca8038670 0x7f9ca803ab20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:33.848 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 --2- 172.21.15.82:0/3377097206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cbc1042f0 0x7f9cbc079b10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 >> 172.21.15.82:0/3377097206 conn(0x7f9cbc0ff9a0 msgr2=0x7f9cbc100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 shutdown_connections 2023-07-14T19:20:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:33.459+0000 7f9cc227f700 1 -- 172.21.15.82:0/3377097206 wait complete. 2023-07-14T19:20:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:mgr is available 2023-07-14T19:20:34.846 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:34 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1158561625' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-14T19:20:34.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:34.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [global] 2023-07-14T19:20:34.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout fsid = 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:34.874 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 2023-07-14T19:20:34.875 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout [osd] 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74ee06c700 1 Processor -- start 2023-07-14T19:20:34.876 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74ee06c700 1 -- start start 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74ee06c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8103e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74ee06c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e81043c0 con 0x7f74e8103a70 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8103e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8103e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39462/0 (socket says 172.21.15.82:39462) 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74e77fe700 1 -- 172.21.15.82:0/1600675266 learned_addr learned my addr 172.21.15.82:0/1600675266 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.260+0000 7f74e77fe700 1 -- 172.21.15.82:0/1600675266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8104500 con 0x7f74e8103a70 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74e77fe700 1 --2- 172.21.15.82:0/1600675266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8103e80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f74d80098a0 tx=0x7f74d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda14c585ac65fe server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:34.877 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74e67fc700 1 -- 172.21.15.82:0/1600675266 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74d8004160 con 0x7f74e8103a70 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74e67fc700 1 -- 172.21.15.82:0/1600675266 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74d80042c0 con 0x7f74e8103a70 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74e67fc700 1 -- 172.21.15.82:0/1600675266 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74d8011a00 con 0x7f74e8103a70 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74ee06c700 1 -- 172.21.15.82:0/1600675266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 msgr2=0x7f74e8103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74ee06c700 1 --2- 172.21.15.82:0/1600675266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8103e80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f74d80098a0 tx=0x7f74d8009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74ee06c700 1 -- 172.21.15.82:0/1600675266 shutdown_connections 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74ee06c700 1 --2- 172.21.15.82:0/1600675266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8103e80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:34.878 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74ee06c700 1 -- 172.21.15.82:0/1600675266 >> 172.21.15.82:0/1600675266 conn(0x7f74e80ff120 msgr2=0x7f74e8101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.261+0000 7f74ee06c700 1 -- 172.21.15.82:0/1600675266 shutdown_connections 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74ee06c700 1 -- 172.21.15.82:0/1600675266 wait complete. 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74ee06c700 1 Processor -- start 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74ee06c700 1 -- start start 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74ee06c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74ee06c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e819bd80 con 0x7f74e8103a70 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:34.879 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39464/0 (socket says 172.21.15.82:39464) 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.262+0000 7f74e77fe700 1 -- 172.21.15.82:0/1158561625 learned_addr learned my addr 172.21.15.82:0/1158561625 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74e77fe700 1 -- 172.21.15.82:0/1158561625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d8009550 con 0x7f74e8103a70 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74e77fe700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8100990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009fb0 tx=0x7f74d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74d8003930 con 0x7f74e8103a70 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74d8003a90 con 0x7f74e8103a70 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e819bec0 con 0x7f74e8103a70 2023-07-14T19:20:34.880 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74d8019610 con 0x7f74e8103a70 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.263+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e819c2e0 con 0x7f74e8103a70 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.264+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f74d80198d0 con 0x7f74e8103a70 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.264+0000 7f74e4ff9700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f74d0038670 0x7f74d003ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.264+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f74d804c490 con 0x7f74e8103a70 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.264+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8191f50 con 0x7f74e8103a70 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.264+0000 7f74e6ffd700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f74d0038670 0x7f74d003ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.265+0000 7f74e6ffd700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f74d0038670 0x7f74d003ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74dc006fd0 tx=0x7f74dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:34.881 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.269+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f74d8019b80 con 0x7f74e8103a70 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.451+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f74e802cd00 con 0x7f74e8103a70 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.452+0000 7f74e4ff9700 1 -- 172.21.15.82:0/1158561625 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f74d8018350 con 0x7f74e8103a70 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f74d0038670 msgr2=0x7f74d003ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f74d0038670 0x7f74d003ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74dc006fd0 tx=0x7f74dc006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 msgr2=0x7f74e8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8100990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009fb0 tx=0x7f74d8011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 shutdown_connections 2023-07-14T19:20:34.882 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f74d0038670 0x7f74d003ab20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:34.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 --2- 172.21.15.82:0/1158561625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74e8103a70 0x7f74e8100990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:34.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 >> 172.21.15.82:0/1158561625 conn(0x7f74e80ff120 msgr2=0x7f74e818f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:34.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 shutdown_connections 2023-07-14T19:20:34.883 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:34.454+0000 7f74ee06c700 1 -- 172.21.15.82:0/1158561625 wait complete. 2023-07-14T19:20:34.883 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling cephadm module... 2023-07-14T19:20:35.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:35 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3220825659' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-14T19:20:35.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: ignoring --setuser ceph since I am not root 2023-07-14T19:20:35.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: ignoring --setgroup ceph since I am not root 2023-07-14T19:20:35.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:35.793+0000 7f76d8f74200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:20:35.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:35.873+0000 7f76d8f74200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:20:36.018 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f37819d2700 1 Processor -- start 2023-07-14T19:20:36.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f37819d2700 1 -- start start 2023-07-14T19:20:36.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f37819d2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c104730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:36.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f37819d2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c104c70 con 0x7f377c104320 2023-07-14T19:20:36.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f377affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:36.019 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f377affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39468/0 (socket says 172.21.15.82:39468) 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f377affd700 1 -- 172.21.15.82:0/1748871026 learned_addr learned my addr 172.21.15.82:0/1748871026 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.324+0000 7f377affd700 1 -- 172.21.15.82:0/1748871026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c104db0 con 0x7f377c104320 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f377affd700 1 --2- 172.21.15.82:0/1748871026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c104730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f376c0098a0 tx=0x7f376c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e06a939368f450 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f377a7fc700 1 -- 172.21.15.82:0/1748871026 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f376c004160 con 0x7f377c104320 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f377a7fc700 1 -- 172.21.15.82:0/1748871026 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f376c0042c0 con 0x7f377c104320 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f377a7fc700 1 -- 172.21.15.82:0/1748871026 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f376c011a00 con 0x7f377c104320 2023-07-14T19:20:36.020 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 -- 172.21.15.82:0/1748871026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 msgr2=0x7f377c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 --2- 172.21.15.82:0/1748871026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c104730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f376c0098a0 tx=0x7f376c009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 -- 172.21.15.82:0/1748871026 shutdown_connections 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 --2- 172.21.15.82:0/1748871026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c104730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 -- 172.21.15.82:0/1748871026 >> 172.21.15.82:0/1748871026 conn(0x7f377c0ff9b0 msgr2=0x7f377c101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 -- 172.21.15.82:0/1748871026 shutdown_connections 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.325+0000 7f37819d2700 1 -- 172.21.15.82:0/1748871026 wait complete. 2023-07-14T19:20:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f37819d2700 1 Processor -- start 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f37819d2700 1 -- start start 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f37819d2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c18f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f37819d2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c18f9e0 con 0x7f377c104320 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f377affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c18f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f377affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c18f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39470/0 (socket says 172.21.15.82:39470) 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.326+0000 7f377affd700 1 -- 172.21.15.82:0/3220825659 learned_addr learned my addr 172.21.15.82:0/3220825659 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f377affd700 1 -- 172.21.15.82:0/3220825659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c009550 con 0x7f377c104320 2023-07-14T19:20:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f377affd700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c18f4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f376c009fb0 tx=0x7f376c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f376c003a40 con 0x7f377c104320 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f376c003ba0 con 0x7f377c104320 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c18daf0 con 0x7f377c104320 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f376c01a750 con 0x7f377c104320 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.327+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c18df10 con 0x7f377c104320 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.328+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f376c01a980 con 0x7f377c104320 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.328+0000 7f37809d0700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f37680386c0 0x7f376803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:36.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.328+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f376c04c320 con 0x7f377c104320 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.328+0000 7f3773fff700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f37680386c0 0x7f376803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.328+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c061c70 con 0x7f377c104320 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.328+0000 7f3773fff700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f37680386c0 0x7f376803ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3764006fd0 tx=0x7f3764006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.333+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f376c01ac30 con 0x7f377c104320 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.572+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f377c18e960 con 0x7f377c104320 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.624+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f376c02d910 con 0x7f377c104320 2023-07-14T19:20:36.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.626+0000 7f37809d0700 1 -- 172.21.15.82:0/3220825659 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f376c0166e0 con 0x7f377c104320 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f37680386c0 msgr2=0x7f376803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f37680386c0 0x7f376803ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3764006fd0 tx=0x7f3764006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 msgr2=0x7f377c18f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c18f4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f376c009fb0 tx=0x7f376c011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 shutdown_connections 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f37680386c0 0x7f376803ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 --2- 172.21.15.82:0/3220825659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f377c104320 0x7f377c18f4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:36.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 >> 172.21.15.82:0/3220825659 conn(0x7f377c0ff9b0 msgr2=0x7f377c101030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:36.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 shutdown_connections 2023-07-14T19:20:36.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:35.628+0000 7f37819d2700 1 -- 172.21.15.82:0/3220825659 wait complete. 2023-07-14T19:20:36.225 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:35.965+0000 7f76d8f74200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:20:36.226 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:36.065+0000 7f76d8f74200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:20:36.561 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:36.242+0000 7f76d8f74200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:20:36.880 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3220825659' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-14T19:20:36.881 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-mon[127461]: mgrmap e5: x(active, since 10s) 2023-07-14T19:20:36.881 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:36.559+0000 7f76d8f74200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:20:36.881 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:36.650+0000 7f76d8f74200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:20:36.882 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:36.782+0000 7f76d8f74200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:20:37.165 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:37.166 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-14T19:20:37.166 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T19:20:37.166 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T19:20:37.166 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T19:20:37.166 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:37.166 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.469+0000 7f2a092f1700 1 Processor -- start 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.469+0000 7f2a092f1700 1 -- start start 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a092f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a041082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a092f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04079d60 con 0x7f2a04105f00 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a041082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a041082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39484/0 (socket says 172.21.15.82:39484) 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a02ffd700 1 -- 172.21.15.82:0/3479611842 learned_addr learned my addr 172.21.15.82:0/3479611842 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:37.167 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a02ffd700 1 -- 172.21.15.82:0/3479611842 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04079ea0 con 0x7f2a04105f00 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a02ffd700 1 --2- 172.21.15.82:0/3479611842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a041082e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098a0 tx=0x7f29f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d2bd61e3d535a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a01ffb700 1 -- 172.21.15.82:0/3479611842 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29f4004160 con 0x7f2a04105f00 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a01ffb700 1 -- 172.21.15.82:0/3479611842 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29f40042c0 con 0x7f2a04105f00 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.470+0000 7f2a01ffb700 1 -- 172.21.15.82:0/3479611842 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29f4011a00 con 0x7f2a04105f00 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 -- 172.21.15.82:0/3479611842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 msgr2=0x7f2a041082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 --2- 172.21.15.82:0/3479611842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a041082e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098a0 tx=0x7f29f4009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:37.168 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 -- 172.21.15.82:0/3479611842 shutdown_connections 2023-07-14T19:20:37.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 --2- 172.21.15.82:0/3479611842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a041082e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:37.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 -- 172.21.15.82:0/3479611842 >> 172.21.15.82:0/3479611842 conn(0x7f2a040ff9a0 msgr2=0x7f2a04101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:37.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 -- 172.21.15.82:0/3479611842 shutdown_connections 2023-07-14T19:20:37.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.471+0000 7f2a092f1700 1 -- 172.21.15.82:0/3479611842 wait complete. 2023-07-14T19:20:37.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a092f1700 1 Processor -- start 2023-07-14T19:20:37.169 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a092f1700 1 -- start start 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a092f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a04199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a092f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04199640 con 0x7f2a04105f00 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a04199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a02ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a04199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39486/0 (socket says 172.21.15.82:39486) 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a02ffd700 1 -- 172.21.15.82:0/3648200618 learned_addr learned my addr 172.21.15.82:0/3648200618 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.472+0000 7f2a02ffd700 1 -- 172.21.15.82:0/3648200618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4009550 con 0x7f2a04105f00 2023-07-14T19:20:37.170 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.473+0000 7f2a02ffd700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a04199100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29f4006af0 tx=0x7f29f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.473+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29f4003a40 con 0x7f2a04105f00 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.473+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a04199840 con 0x7f2a04105f00 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.473+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f29f4003ba0 con 0x7f2a04105f00 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.473+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29f401a700 con 0x7f2a04105f00 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.474+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a04199c60 con 0x7f2a04105f00 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.474+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f29f4024ad0 con 0x7f2a04105f00 2023-07-14T19:20:37.171 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.474+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a04061c70 con 0x7f2a04105f00 2023-07-14T19:20:37.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.475+0000 7f29ebfff700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 0x7f29ec03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:37.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.475+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f29f404c710 con 0x7f2a04105f00 2023-07-14T19:20:37.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.475+0000 7f2a027fc700 1 -- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 msgr2=0x7f29ec03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:37.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.475+0000 7f2a027fc700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 0x7f29ec03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:20:37.172 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.479+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f29f4018d20 con 0x7f2a04105f00 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.676+0000 7f2a027fc700 1 -- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 msgr2=0x7f29ec03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.676+0000 7f2a027fc700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 0x7f29ec03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.724+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f2a0419cde0 con 0x7f2a04105f00 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.724+0000 7f29ebfff700 1 -- 172.21.15.82:0/3648200618 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f29f4018b40 con 0x7f2a04105f00 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.726+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 msgr2=0x7f29ec03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.726+0000 7f2a092f1700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 0x7f29ec03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.726+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 msgr2=0x7f2a04199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:37.173 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.726+0000 7f2a092f1700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a04199100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29f4006af0 tx=0x7f29f4011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:37.174 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.727+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 shutdown_connections 2023-07-14T19:20:37.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.727+0000 7f2a092f1700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7f29ec038710 0x7f29ec03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:37.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.727+0000 7f2a092f1700 1 --2- 172.21.15.82:0/3648200618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a04105f00 0x7f2a04199100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:37.176 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.727+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 >> 172.21.15.82:0/3648200618 conn(0x7f2a040ff9a0 msgr2=0x7f2a04100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:37.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.727+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 shutdown_connections 2023-07-14T19:20:37.177 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:36.727+0000 7f2a092f1700 1 -- 172.21.15.82:0/3648200618 wait complete. 2023-07-14T19:20:37.177 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2023-07-14T19:20:37.177 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 5... 2023-07-14T19:20:37.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:36.884+0000 7f76d8f74200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:20:37.631 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:37.321+0000 7f76d8f74200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:20:37.910 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:37 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3648200618' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T19:20:38.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:37.908+0000 7f76d8f74200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:20:38.215 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:38.075+0000 7f76d8f74200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:20:39.090 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:38.831+0000 7f76d8f74200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:20:39.090 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:38.920+0000 7f76d8f74200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:20:39.363 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:39.089+0000 7f76d8f74200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:20:39.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:39.362+0000 7f76d8f74200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:20:39.715 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:39.594+0000 7f76d8f74200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:20:39.997 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:39.740+0000 7f76d8f74200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:20:40.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:39.996+0000 7f76d8f74200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:20:40.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:40.685+0000 7f76d8f74200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:20:40.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:40.813+0000 7f76d8f74200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:20:41.327 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:41.001+0000 7f76d8f74200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:20:41.327 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:41.113+0000 7f76d8f74200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:20:41.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:41.326+0000 7f76d8f74200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:20:42.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:42.168+0000 7f76d8f74200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:20:43.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: Active manager daemon x restarted 2023-07-14T19:20:43.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: Activating manager daemon x 2023-07-14T19:20:43.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: osdmap e2: 0 total, 0 up, 0 in 2023-07-14T19:20:43.329 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: mgrmap e6: x(active, starting, since 0.00323847s) 2023-07-14T19:20:43.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:20:43.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T19:20:43.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T19:20:43.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T19:20:43.330 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T19:20:43.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: Manager daemon x is now available 2023-07-14T19:20:43.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: Found migration_current of "None". Setting to last migration. 2023-07-14T19:20:43.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:43.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:43.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:20:43.331 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:20:43.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:20:43.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T19:20:43.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T19:20:43.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:43.332 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:43 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:43.620 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:43.620 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-14T19:20:43.620 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T19:20:43.621 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:43.621 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93f615700 1 Processor -- start 2023-07-14T19:20:43.621 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93f615700 1 -- start start 2023-07-14T19:20:43.622 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93f615700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff938104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:43.622 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93f615700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938104c40 con 0x7ff9381042f0 2023-07-14T19:20:43.623 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93d3b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff938104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:43.623 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93d3b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff938104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39494/0 (socket says 172.21.15.82:39494) 2023-07-14T19:20:43.623 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93d3b1700 1 -- 172.21.15.82:0/3165624231 learned_addr learned my addr 172.21.15.82:0/3165624231 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:43.624 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.613+0000 7ff93d3b1700 1 -- 172.21.15.82:0/3165624231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938104d80 con 0x7ff9381042f0 2023-07-14T19:20:43.624 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93d3b1700 1 --2- 172.21.15.82:0/3165624231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff938104700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff9340098a0 tx=0x7ff934009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93efeae2888e3043 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:43.625 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff92ffff700 1 -- 172.21.15.82:0/3165624231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff934004160 con 0x7ff9381042f0 2023-07-14T19:20:43.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff92ffff700 1 -- 172.21.15.82:0/3165624231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9340042c0 con 0x7ff9381042f0 2023-07-14T19:20:43.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff92ffff700 1 -- 172.21.15.82:0/3165624231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff934011a00 con 0x7ff9381042f0 2023-07-14T19:20:43.626 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93f615700 1 -- 172.21.15.82:0/3165624231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 msgr2=0x7ff938104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:43.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93f615700 1 --2- 172.21.15.82:0/3165624231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff938104700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff9340098a0 tx=0x7ff934009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93f615700 1 -- 172.21.15.82:0/3165624231 shutdown_connections 2023-07-14T19:20:43.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93f615700 1 --2- 172.21.15.82:0/3165624231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff938104700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.627 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93f615700 1 -- 172.21.15.82:0/3165624231 >> 172.21.15.82:0/3165624231 conn(0x7ff9380ff9a0 msgr2=0x7ff938101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:43.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.614+0000 7ff93f615700 1 -- 172.21.15.82:0/3165624231 shutdown_connections 2023-07-14T19:20:43.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93f615700 1 -- 172.21.15.82:0/3165624231 wait complete. 2023-07-14T19:20:43.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93f615700 1 Processor -- start 2023-07-14T19:20:43.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93f615700 1 -- start start 2023-07-14T19:20:43.628 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93f615700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff93818f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:43.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93f615700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93818f9b0 con 0x7ff9381042f0 2023-07-14T19:20:43.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93d3b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff93818f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:43.629 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93d3b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff93818f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39496/0 (socket says 172.21.15.82:39496) 2023-07-14T19:20:43.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.615+0000 7ff93d3b1700 1 -- 172.21.15.82:0/3233083798 learned_addr learned my addr 172.21.15.82:0/3233083798 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:43.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff93d3b1700 1 -- 172.21.15.82:0/3233083798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff934009550 con 0x7ff9381042f0 2023-07-14T19:20:43.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff93d3b1700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff93818f470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff934006af0 tx=0x7ff934011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:43.630 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff934003a40 con 0x7ff9381042f0 2023-07-14T19:20:43.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff934003ba0 con 0x7ff9381042f0 2023-07-14T19:20:43.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93818dac0 con 0x7ff9381042f0 2023-07-14T19:20:43.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff93401a6e0 con 0x7ff9381042f0 2023-07-14T19:20:43.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.616+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93818dee0 con 0x7ff9381042f0 2023-07-14T19:20:43.631 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.617+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff93401a840 con 0x7ff9381042f0 2023-07-14T19:20:43.632 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.617+0000 7ff92e7fc700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:43.632 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.617+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 --> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff938061c70 con 0x7ff924038760 2023-07-14T19:20:43.632 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.617+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff93404c8f0 con 0x7ff9381042f0 2023-07-14T19:20:43.633 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.617+0000 7ff93cbb0700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 msgr2=0x7ff92403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:43.633 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.617+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:20:43.633 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.818+0000 7ff93cbb0700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 msgr2=0x7ff92403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:43.633 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:37.818+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:20:43.633 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:38.219+0000 7ff93cbb0700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 msgr2=0x7ff92403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:43.633 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:38.219+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:39.021+0000 7ff93cbb0700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 msgr2=0x7ff92403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:39.021+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:40.622+0000 7ff93cbb0700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 msgr2=0x7ff92403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/3383156270 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:40.622+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:42.184+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7ff93401acd0 con 0x7ff9381042f0 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:42.184+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 msgr2=0x7ff92403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:42.184+0000 7ff92e7fc700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.634 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.185+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7ff93401bcf0 con 0x7ff9381042f0 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.186+0000 7ff92e7fc700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7ff92403be80 0x7ff92403e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.186+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff938061c70 con 0x7ff92403be80 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.186+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7ff92403be80 0x7ff92403e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.186+0000 7ff93cbb0700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7ff92403be80 0x7ff92403e260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff928003a10 tx=0x7ff9280092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.187+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff938061c70 con 0x7ff92403be80 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff93818f010 con 0x7ff92403be80 2023-07-14T19:20:43.635 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff92e7fc700 1 -- 172.21.15.82:0/3233083798 <== mgr.14120 v2:172.21.15.82:6800/1035467551 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff93818f010 con 0x7ff92403be80 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7ff92403be80 msgr2=0x7ff92403e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7ff92403be80 0x7ff92403e260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff928003a10 tx=0x7ff9280092b0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 msgr2=0x7ff93818f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff93818f470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff934006af0 tx=0x7ff934011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 shutdown_connections 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.193+0000 7ff93f615700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7ff92403be80 0x7ff92403e260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.636 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.194+0000 7ff93f615700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9381042f0 0x7ff93818f470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.194+0000 7ff93f615700 1 --2- 172.21.15.82:0/3233083798 >> [v2:172.21.15.82:6800/3383156270,v1:172.21.15.82:6801/3383156270] conn(0x7ff924038760 0x7ff92403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:43.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.194+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 >> 172.21.15.82:0/3233083798 conn(0x7ff9380ff9a0 msgr2=0x7ff9380798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:43.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.194+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 shutdown_connections 2023-07-14T19:20:43.637 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:43.194+0000 7ff93f615700 1 -- 172.21.15.82:0/3233083798 wait complete. 2023-07-14T19:20:43.637 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 5 is available 2023-07-14T19:20:43.638 INFO:teuthology.orchestra.run.smithi082.stdout:Setting orchestrator backend to cephadm... 2023-07-14T19:20:44.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T19:20:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: mgrmap e7: x(active, since 1.00596s) 2023-07-14T19:20:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T19:20:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: [14/Jul/2023:19:20:43] ENGINE Bus STARTING 2023-07-14T19:20:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: [14/Jul/2023:19:20:43] ENGINE Serving on http://172.21.15.82:8765 2023-07-14T19:20:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: [14/Jul/2023:19:20:43] ENGINE Serving on https://172.21.15.82:7150 2023-07-14T19:20:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: [14/Jul/2023:19:20:43] ENGINE Bus STARTED 2023-07-14T19:20:44.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:44 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:20:44.659 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.105+0000 7f28633e0700 1 Processor -- start 2023-07-14T19:20:44.660 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f28633e0700 1 -- start start 2023-07-14T19:20:44.660 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f28633e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:44.660 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f28633e0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c104c40 con 0x7f285c1042f0 2023-07-14T19:20:44.660 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f286117c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:44.660 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f286117c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39548/0 (socket says 172.21.15.82:39548) 2023-07-14T19:20:44.660 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f286117c700 1 -- 172.21.15.82:0/1382071048 learned_addr learned my addr 172.21.15.82:0/1382071048 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f286117c700 1 -- 172.21.15.82:0/1382071048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c104d80 con 0x7f285c1042f0 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.106+0000 7f286117c700 1 --2- 172.21.15.82:0/1382071048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c104700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f284c0098a0 tx=0x7f284c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb218b1ce209551 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f2853fff700 1 -- 172.21.15.82:0/1382071048 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f284c004160 con 0x7f285c1042f0 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f2853fff700 1 -- 172.21.15.82:0/1382071048 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f284c0042c0 con 0x7f285c1042f0 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f2853fff700 1 -- 172.21.15.82:0/1382071048 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f284c011a00 con 0x7f285c1042f0 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 -- 172.21.15.82:0/1382071048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 msgr2=0x7f285c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:44.661 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 --2- 172.21.15.82:0/1382071048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c104700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f284c0098a0 tx=0x7f284c009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 -- 172.21.15.82:0/1382071048 shutdown_connections 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 --2- 172.21.15.82:0/1382071048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c104700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 -- 172.21.15.82:0/1382071048 >> 172.21.15.82:0/1382071048 conn(0x7f285c0ff9a0 msgr2=0x7f285c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 -- 172.21.15.82:0/1382071048 shutdown_connections 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.107+0000 7f28633e0700 1 -- 172.21.15.82:0/1382071048 wait complete. 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f28633e0700 1 Processor -- start 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f28633e0700 1 -- start start 2023-07-14T19:20:44.662 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f28633e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c18f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f28633e0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c18f9b0 con 0x7f285c1042f0 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f286117c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c18f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f286117c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c18f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39550/0 (socket says 172.21.15.82:39550) 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f286117c700 1 -- 172.21.15.82:0/19878377 learned_addr learned my addr 172.21.15.82:0/19878377 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f286117c700 1 -- 172.21.15.82:0/19878377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c009550 con 0x7f285c1042f0 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.108+0000 7f286117c700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c18f470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f284c009fb0 tx=0x7f284c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.109+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f284c003930 con 0x7f285c1042f0 2023-07-14T19:20:44.663 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.109+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f284c003a90 con 0x7f285c1042f0 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.109+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c18dac0 con 0x7f285c1042f0 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.109+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f284c019610 con 0x7f285c1042f0 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.109+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c18dee0 con 0x7f285c1042f0 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.110+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f284c0198d0 con 0x7f285c1042f0 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.110+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c061c70 con 0x7f285c1042f0 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.110+0000 7f28527fc700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f28480385a0 0x7f284803aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:44.664 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.110+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f284c04c880 con 0x7f285c1042f0 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.110+0000 7f286097b700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f28480385a0 0x7f284803aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.110+0000 7f286097b700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f28480385a0 0x7f284803aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2858006fd0 tx=0x7f2858006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.115+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f284c02b340 con 0x7f285c1042f0 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.317+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f285c101b00 con 0x7f28480385a0 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.323+0000 7f28527fc700 1 -- 172.21.15.82:0/19878377 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f285c101b00 con 0x7f28480385a0 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f28480385a0 msgr2=0x7f284803aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f28480385a0 0x7f284803aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2858006fd0 tx=0x7f2858006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:44.665 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 msgr2=0x7f285c18f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c18f470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f284c009fb0 tx=0x7f284c011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 shutdown_connections 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f28480385a0 0x7f284803aa50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 --2- 172.21.15.82:0/19878377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285c1042f0 0x7f285c18f470 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 >> 172.21.15.82:0/19878377 conn(0x7f285c0ff9a0 msgr2=0x7f285c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 shutdown_connections 2023-07-14T19:20:44.666 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:44.325+0000 7f28633e0700 1 -- 172.21.15.82:0/19878377 wait complete. 2023-07-14T19:20:45.602 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-14T19:20:45.603 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f176c072700 1 Processor -- start 2023-07-14T19:20:45.603 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f176c072700 1 -- start start 2023-07-14T19:20:45.603 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f176c072700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f1764108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:45.603 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f176c072700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764079d20 con 0x7f1764105f40 2023-07-14T19:20:45.603 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f1769e0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f1764108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f1769e0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f1764108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39554/0 (socket says 172.21.15.82:39554) 2023-07-14T19:20:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.085+0000 7f1769e0e700 1 -- 172.21.15.82:0/3052720483 learned_addr learned my addr 172.21.15.82:0/3052720483 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f1769e0e700 1 -- 172.21.15.82:0/3052720483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764079e60 con 0x7f1764105f40 2023-07-14T19:20:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f1769e0e700 1 --2- 172.21.15.82:0/3052720483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f1764108320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17600098a0 tx=0x7f1760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed13d9933016e2cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f1768e0c700 1 -- 172.21.15.82:0/3052720483 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1760004160 con 0x7f1764105f40 2023-07-14T19:20:45.604 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f1768e0c700 1 -- 172.21.15.82:0/3052720483 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17600042c0 con 0x7f1764105f40 2023-07-14T19:20:45.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f1768e0c700 1 -- 172.21.15.82:0/3052720483 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1760011a00 con 0x7f1764105f40 2023-07-14T19:20:45.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f176c072700 1 -- 172.21.15.82:0/3052720483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 msgr2=0x7f1764108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:45.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.086+0000 7f176c072700 1 --2- 172.21.15.82:0/3052720483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f1764108320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f17600098a0 tx=0x7f1760009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:45.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 -- 172.21.15.82:0/3052720483 shutdown_connections 2023-07-14T19:20:45.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 --2- 172.21.15.82:0/3052720483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f1764108320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:45.605 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 -- 172.21.15.82:0/3052720483 >> 172.21.15.82:0/3052720483 conn(0x7f17640ff9a0 msgr2=0x7f1764101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 -- 172.21.15.82:0/3052720483 shutdown_connections 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 -- 172.21.15.82:0/3052720483 wait complete. 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 Processor -- start 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 -- start start 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f17641058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f176c072700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764105de0 con 0x7f1764105f40 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.087+0000 7f1769e0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f17641058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:45.606 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f1769e0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f17641058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39556/0 (socket says 172.21.15.82:39556) 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f1769e0e700 1 -- 172.21.15.82:0/3425151650 learned_addr learned my addr 172.21.15.82:0/3425151650 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f1769e0e700 1 -- 172.21.15.82:0/3425151650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760009550 con 0x7f1764105f40 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f1769e0e700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f17641058a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1760006af0 tx=0x7f1760011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1760003a40 con 0x7f1764105f40 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1764103f50 con 0x7f1764105f40 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1760003ba0 con 0x7f1764105f40 2023-07-14T19:20:45.607 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f176001a700 con 0x7f1764105f40 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.088+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17641043d0 con 0x7f1764105f40 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.089+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f1760024430 con 0x7f1764105f40 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.089+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764109460 con 0x7f1764105f40 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.089+0000 7f175affd700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f17500385e0 0x7f175003aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.089+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f176004c520 con 0x7f1764105f40 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.089+0000 7f176960d700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f17500385e0 0x7f175003aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:45.608 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.090+0000 7f176960d700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f17500385e0 0x7f175003aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1754006fd0 tx=0x7f1754006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.094+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1760018b80 con 0x7f1764105f40 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.187+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f176002d930 con 0x7f1764105f40 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.281+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f1764101c90 con 0x7f17500385e0 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.281+0000 7f175affd700 1 -- 172.21.15.82:0/3425151650 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f1764101c90 con 0x7f17500385e0 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f17500385e0 msgr2=0x7f175003aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f17500385e0 0x7f175003aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1754006fd0 tx=0x7f1754006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 msgr2=0x7f17641058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:45.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f17641058a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1760006af0 tx=0x7f1760011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:45.610 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 shutdown_connections 2023-07-14T19:20:45.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f17500385e0 0x7f175003aa90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:45.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 --2- 172.21.15.82:0/3425151650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764105f40 0x7f17641058a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:45.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.283+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 >> 172.21.15.82:0/3425151650 conn(0x7f17640ff9a0 msgr2=0x7f1764101070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:45.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.284+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 shutdown_connections 2023-07-14T19:20:45.611 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:45.284+0000 7f176c072700 1 -- 172.21.15.82:0/3425151650 wait complete. 2023-07-14T19:20:45.612 INFO:teuthology.orchestra.run.smithi082.stdout:Generating ssh key... 2023-07-14T19:20:45.612 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:45 smithi082 ceph-mon[127461]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:45.613 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:45 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:45.613 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:45 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:20:45.613 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:45 smithi082 ceph-mon[127461]: mgrmap e8: x(active, since 3s) 2023-07-14T19:20:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:46 smithi082 ceph-mon[127461]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:47.323 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: Generating public/private rsa key pair. 2023-07-14T19:20:47.323 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: Your identification has been saved in /tmp/tmpgq0eufq6/key. 2023-07-14T19:20:47.323 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: Your public key has been saved in /tmp/tmpgq0eufq6/key.pub. 2023-07-14T19:20:47.324 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: The key fingerprint is: 2023-07-14T19:20:47.324 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: SHA256:CkT0j1+lHu14hmPM2Bky1HlFAiZDluUMARcBk23phkQ ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:47.324 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: The key's randomart image is: 2023-07-14T19:20:47.324 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: +---[RSA 3072]----+ 2023-07-14T19:20:47.324 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | .o.E*@B+...o | 2023-07-14T19:20:47.324 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | . .o+=O . o | 2023-07-14T19:20:47.325 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | ...+. = o | 2023-07-14T19:20:47.325 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | . .+o = | 2023-07-14T19:20:47.325 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | . ..S = . | 2023-07-14T19:20:47.325 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | . o X B | 2023-07-14T19:20:47.325 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | . o @ + | 2023-07-14T19:20:47.325 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | . + | 2023-07-14T19:20:47.326 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: | | 2023-07-14T19:20:47.326 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: +----[SHA256]-----+ 2023-07-14T19:20:47.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b8298a700 1 Processor -- start 2023-07-14T19:20:47.448 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b8298a700 1 -- start start 2023-07-14T19:20:47.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b8298a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:47.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b8298a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c104030 con 0x7f7b7c1036e0 2023-07-14T19:20:47.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:47.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39560/0 (socket says 172.21.15.82:39560) 2023-07-14T19:20:47.449 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b7bfff700 1 -- 172.21.15.82:0/1721041531 learned_addr learned my addr 172.21.15.82:0/1721041531 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.057+0000 7f7b7bfff700 1 -- 172.21.15.82:0/1721041531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c104170 con 0x7f7b7c1036e0 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b7bfff700 1 --2- 172.21.15.82:0/1721041531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c103af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0098a0 tx=0x7f7b6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33afe65ef3bbecc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b7affd700 1 -- 172.21.15.82:0/1721041531 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b6c004160 con 0x7f7b7c1036e0 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b7affd700 1 -- 172.21.15.82:0/1721041531 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b6c0042c0 con 0x7f7b7c1036e0 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b7affd700 1 -- 172.21.15.82:0/1721041531 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b6c011a00 con 0x7f7b7c1036e0 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 -- 172.21.15.82:0/1721041531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 msgr2=0x7f7b7c103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:47.450 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 --2- 172.21.15.82:0/1721041531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c103af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0098a0 tx=0x7f7b6c009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:47.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 -- 172.21.15.82:0/1721041531 shutdown_connections 2023-07-14T19:20:47.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 --2- 172.21.15.82:0/1721041531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c103af0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:47.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 -- 172.21.15.82:0/1721041531 >> 172.21.15.82:0/1721041531 conn(0x7f7b7c0ff120 msgr2=0x7f7b7c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:47.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 -- 172.21.15.82:0/1721041531 shutdown_connections 2023-07-14T19:20:47.451 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.058+0000 7f7b8298a700 1 -- 172.21.15.82:0/1721041531 wait complete. 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b8298a700 1 Processor -- start 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b8298a700 1 -- start start 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b8298a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b8298a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c199650 con 0x7f7b7c1036e0 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39562/0 (socket says 172.21.15.82:39562) 2023-07-14T19:20:47.452 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.059+0000 7f7b7bfff700 1 -- 172.21.15.82:0/66738685 learned_addr learned my addr 172.21.15.82:0/66738685 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b7bfff700 1 -- 172.21.15.82:0/66738685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009550 con 0x7f7b7c1036e0 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b7bfff700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c199110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009fb0 tx=0x7f7b6c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b6c003930 con 0x7f7b7c1036e0 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c199850 con 0x7f7b7c1036e0 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b6c003a90 con 0x7f7b7c1036e0 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b6c0195e0 con 0x7f7b7c1036e0 2023-07-14T19:20:47.453 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.060+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c199c70 con 0x7f7b7c1036e0 2023-07-14T19:20:47.454 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.061+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7b6c0198a0 con 0x7f7b7c1036e0 2023-07-14T19:20:47.454 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.061+0000 7f7b797fa700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f7b640386c0 0x7f7b6403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:47.454 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.061+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7c061c70 con 0x7f7b7c1036e0 2023-07-14T19:20:47.454 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.061+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b6c04c860 con 0x7f7b7c1036e0 2023-07-14T19:20:47.454 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.061+0000 7f7b7b7fe700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f7b640386c0 0x7f7b6403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.061+0000 7f7b7b7fe700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f7b640386c0 0x7f7b6403ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7b70006fd0 tx=0x7f7b70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.066+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7b6c02b340 con 0x7f7b7c1036e0 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:46.251+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b7c1005d0 con 0x7f7b640386c0 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.063+0000 7f7b797fa700 1 -- 172.21.15.82:0/66738685 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7b7c1005d0 con 0x7f7b640386c0 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f7b640386c0 msgr2=0x7f7b6403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f7b640386c0 0x7f7b6403ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7b70006fd0 tx=0x7f7b70006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 msgr2=0x7f7b7c199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:47.455 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c199110 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009fb0 tx=0x7f7b6c011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:47.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 shutdown_connections 2023-07-14T19:20:47.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f7b640386c0 0x7f7b6403ab70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:47.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 --2- 172.21.15.82:0/66738685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b7c1036e0 0x7f7b7c199110 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:47.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 >> 172.21.15.82:0/66738685 conn(0x7f7b7c0ff120 msgr2=0x7f7b7c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:47.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 shutdown_connections 2023-07-14T19:20:47.456 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.065+0000 7f7b8298a700 1 -- 172.21.15.82:0/66738685 wait complete. 2023-07-14T19:20:47.617 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-mon[127461]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:47.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-mon[127461]: Generating ssh key... 2023-07-14T19:20:47.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:47.618 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:47 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:48.388 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:20:48.389 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74f089e700 1 Processor -- start 2023-07-14T19:20:48.389 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74f089e700 1 -- start start 2023-07-14T19:20:48.389 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74f089e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec103af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:48.389 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74f089e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec104030 con 0x7f74ec1036e0 2023-07-14T19:20:48.389 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:48.389 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39566/0 (socket says 172.21.15.82:39566) 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.851+0000 7f74ea59c700 1 -- 172.21.15.82:0/3725681776 learned_addr learned my addr 172.21.15.82:0/3725681776 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74ea59c700 1 -- 172.21.15.82:0/3725681776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec104170 con 0x7f74ec1036e0 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74ea59c700 1 --2- 172.21.15.82:0/3725681776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec103af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f74e00098a0 tx=0x7f74e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92307ad390e01d22 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74e959a700 1 -- 172.21.15.82:0/3725681776 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e0004160 con 0x7f74ec1036e0 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74e959a700 1 -- 172.21.15.82:0/3725681776 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74e00042c0 con 0x7f74ec1036e0 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74e959a700 1 -- 172.21.15.82:0/3725681776 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e0011a00 con 0x7f74ec1036e0 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74f089e700 1 -- 172.21.15.82:0/3725681776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 msgr2=0x7f74ec103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:48.390 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.852+0000 7f74f089e700 1 --2- 172.21.15.82:0/3725681776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec103af0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f74e00098a0 tx=0x7f74e0009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 -- 172.21.15.82:0/3725681776 shutdown_connections 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 --2- 172.21.15.82:0/3725681776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec103af0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 -- 172.21.15.82:0/3725681776 >> 172.21.15.82:0/3725681776 conn(0x7f74ec0ff120 msgr2=0x7f74ec101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 -- 172.21.15.82:0/3725681776 shutdown_connections 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 -- 172.21.15.82:0/3725681776 wait complete. 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 Processor -- start 2023-07-14T19:20:48.391 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.853+0000 7f74f089e700 1 -- start start 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74f089e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec106070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74f089e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1065b0 con 0x7f74ec1036e0 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec106070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74ea59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec106070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39568/0 (socket says 172.21.15.82:39568) 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74ea59c700 1 -- 172.21.15.82:0/3663179837 learned_addr learned my addr 172.21.15.82:0/3663179837 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74ea59c700 1 -- 172.21.15.82:0/3663179837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e0009550 con 0x7f74ec1036e0 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74ea59c700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec106070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009fb0 tx=0x7f74e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:48.392 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e0003930 con 0x7f74ec1036e0 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74e0003a90 con 0x7f74ec1036e0 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec1046c0 con 0x7f74ec1036e0 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74e00195e0 con 0x7f74ec1036e0 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.854+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec104ae0 con 0x7f74ec1036e0 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.855+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f74e0019850 con 0x7f74ec1036e0 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.855+0000 7f74db7fe700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f74d4038710 0x7f74d403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:48.393 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.855+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f74e0003c00 con 0x7f74ec1036e0 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.855+0000 7f74e9d9b700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f74d4038710 0x7f74d403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.855+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec061c70 con 0x7f74ec1036e0 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.856+0000 7f74e9d9b700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f74d4038710 0x7f74d403abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f74dc006fd0 tx=0x7f74dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:47.860+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f74e0019b50 con 0x7f74ec1036e0 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.046+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f74ec105500 con 0x7f74d4038710 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.046+0000 7f74db7fe700 1 -- 172.21.15.82:0/3663179837 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f74ec105500 con 0x7f74d4038710 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f74d4038710 msgr2=0x7f74d403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:48.394 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f74d4038710 0x7f74d403abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f74dc006fd0 tx=0x7f74dc006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 msgr2=0x7f74ec106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec106070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009fb0 tx=0x7f74e0011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 shutdown_connections 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f74d4038710 0x7f74d403abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 --2- 172.21.15.82:0/3663179837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1036e0 0x7f74ec106070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 >> 172.21.15.82:0/3663179837 conn(0x7f74ec0ff120 msgr2=0x7f74ec0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 shutdown_connections 2023-07-14T19:20:48.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.048+0000 7f74f089e700 1 -- 172.21.15.82:0/3663179837 wait complete. 2023-07-14T19:20:48.396 INFO:teuthology.orchestra.run.smithi082.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-14T19:20:48.396 INFO:teuthology.orchestra.run.smithi082.stdout:Adding key to root@localhost authorized_keys... 2023-07-14T19:20:48.396 INFO:teuthology.orchestra.run.smithi082.stdout:Adding host smithi082... 2023-07-14T19:20:48.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:48 smithi082 ceph-mon[127461]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:49.862 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:49 smithi082 ceph-mon[127461]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:49.863 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:49 smithi082 ceph-mon[127461]: Deploying cephadm binary to smithi082 2023-07-14T19:20:52.039 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Added host 'smithi082' with addr '172.21.15.82' 2023-07-14T19:20:52.039 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f497a8700 1 Processor -- start 2023-07-14T19:20:52.039 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f497a8700 1 -- start start 2023-07-14T19:20:52.039 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f497a8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:52.039 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f497a8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44107360 con 0x7f5f44106a10 2023-07-14T19:20:52.039 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f42ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f42ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39572/0 (socket says 172.21.15.82:39572) 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f42ffd700 1 -- 172.21.15.82:0/3989991147 learned_addr learned my addr 172.21.15.82:0/3989991147 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f42ffd700 1 -- 172.21.15.82:0/3989991147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f441074a0 con 0x7f5f44106a10 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.865+0000 7f5f42ffd700 1 --2- 172.21.15.82:0/3989991147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098a0 tx=0x7f5f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0409a7001cf54a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f41ffb700 1 -- 172.21.15.82:0/3989991147 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f34004160 con 0x7f5f44106a10 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f41ffb700 1 -- 172.21.15.82:0/3989991147 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f340042c0 con 0x7f5f44106a10 2023-07-14T19:20:52.040 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f41ffb700 1 -- 172.21.15.82:0/3989991147 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f34011a00 con 0x7f5f44106a10 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 -- 172.21.15.82:0/3989991147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 msgr2=0x7f5f44106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 --2- 172.21.15.82:0/3989991147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106e20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098a0 tx=0x7f5f34009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 -- 172.21.15.82:0/3989991147 shutdown_connections 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 --2- 172.21.15.82:0/3989991147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106e20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 -- 172.21.15.82:0/3989991147 >> 172.21.15.82:0/3989991147 conn(0x7f5f440ff120 msgr2=0x7f5f44101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 -- 172.21.15.82:0/3989991147 shutdown_connections 2023-07-14T19:20:52.041 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.866+0000 7f5f497a8700 1 -- 172.21.15.82:0/3989991147 wait complete. 2023-07-14T19:20:52.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f497a8700 1 Processor -- start 2023-07-14T19:20:52.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f497a8700 1 -- start start 2023-07-14T19:20:52.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f497a8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:52.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f497a8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44106650 con 0x7f5f44106a10 2023-07-14T19:20:52.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f42ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:52.042 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f42ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39574/0 (socket says 172.21.15.82:39574) 2023-07-14T19:20:52.043 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.867+0000 7f5f42ffd700 1 -- 172.21.15.82:0/1922057847 learned_addr learned my addr 172.21.15.82:0/1922057847 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:52.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f42ffd700 1 -- 172.21.15.82:0/1922057847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009550 con 0x7f5f44106a10 2023-07-14T19:20:52.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f42ffd700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009fb0 tx=0x7f5f34011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:52.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f34003930 con 0x7f5f44106a10 2023-07-14T19:20:52.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f44104760 con 0x7f5f44106a10 2023-07-14T19:20:52.045 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f34003a90 con 0x7f5f44106a10 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f34019610 con 0x7f5f44106a10 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.868+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f44104b80 con 0x7f5f44106a10 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.869+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5f340198f0 con 0x7f5f44106a10 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.869+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f44061ce0 con 0x7f5f44106a10 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.869+0000 7f5f2bfff700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f5f2c0386c0 0x7f5f2c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.869+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5f3404c860 con 0x7f5f44106a10 2023-07-14T19:20:52.046 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.869+0000 7f5f427fc700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f5f2c0386c0 0x7f5f2c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.869+0000 7f5f427fc700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f5f2c0386c0 0x7f5f2c03ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f38006fd0 tx=0x7f5f38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:48.873+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5f44061ce0 con 0x7f5f44106a10 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:49.062+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "addr": "172.21.15.82", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f44061ce0 con 0x7f5f2c0386c0 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.654+0000 7f5f2bfff700 1 -- 172.21.15.82:0/1922057847 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f5f44061ce0 con 0x7f5f2c0386c0 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.656+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f5f2c0386c0 msgr2=0x7f5f2c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.656+0000 7f5f497a8700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f5f2c0386c0 0x7f5f2c03ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f38006fd0 tx=0x7f5f38006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 msgr2=0x7f5f44106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:52.047 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106110 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009fb0 tx=0x7f5f34011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 shutdown_connections 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f5f2c0386c0 0x7f5f2c03ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 --2- 172.21.15.82:0/1922057847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f44106a10 0x7f5f44106110 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 >> 172.21.15.82:0/1922057847 conn(0x7f5f440ff120 msgr2=0x7f5f441007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 shutdown_connections 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:51.657+0000 7f5f497a8700 1 -- 172.21.15.82:0/1922057847 wait complete. 2023-07-14T19:20:52.048 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mon service... 2023-07-14T19:20:52.910 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:52 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:52.910 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:52 smithi082 ceph-mon[127461]: Added host smithi082 2023-07-14T19:20:52.911 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:52 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:20:53.187 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-14T19:20:53.188 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.537+0000 7faf5cdd3700 1 Processor -- start 2023-07-14T19:20:53.188 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.537+0000 7faf5cdd3700 1 -- start start 2023-07-14T19:20:53.188 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5cdd3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf58108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5cdd3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58079d20 con 0x7faf58105f40 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf58108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf58108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39608/0 (socket says 172.21.15.82:39608) 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5659c700 1 -- 172.21.15.82:0/1729379608 learned_addr learned my addr 172.21.15.82:0/1729379608 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5659c700 1 -- 172.21.15.82:0/1729379608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58079e60 con 0x7faf58105f40 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.538+0000 7faf5659c700 1 --2- 172.21.15.82:0/1729379608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf58108320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0098a0 tx=0x7faf4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b80594fa0cf2595 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:53.189 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5559a700 1 -- 172.21.15.82:0/1729379608 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf4c004160 con 0x7faf58105f40 2023-07-14T19:20:53.190 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5559a700 1 -- 172.21.15.82:0/1729379608 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf4c0042c0 con 0x7faf58105f40 2023-07-14T19:20:53.190 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5559a700 1 -- 172.21.15.82:0/1729379608 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf4c011a00 con 0x7faf58105f40 2023-07-14T19:20:53.190 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1729379608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 msgr2=0x7faf58108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:53.190 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 --2- 172.21.15.82:0/1729379608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf58108320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0098a0 tx=0x7faf4c009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:53.190 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1729379608 shutdown_connections 2023-07-14T19:20:53.190 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 --2- 172.21.15.82:0/1729379608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf58108320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:53.191 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1729379608 >> 172.21.15.82:0/1729379608 conn(0x7faf580ff9a0 msgr2=0x7faf58101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:53.191 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1729379608 shutdown_connections 2023-07-14T19:20:53.191 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.539+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1729379608 wait complete. 2023-07-14T19:20:53.191 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.540+0000 7faf5cdd3700 1 Processor -- start 2023-07-14T19:20:53.191 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.540+0000 7faf5cdd3700 1 -- start start 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.540+0000 7faf5cdd3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf581990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.540+0000 7faf5cdd3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf581995e0 con 0x7faf58105f40 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.540+0000 7faf5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf581990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.540+0000 7faf5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf581990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39610/0 (socket says 172.21.15.82:39610) 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf5659c700 1 -- 172.21.15.82:0/1216077964 learned_addr learned my addr 172.21.15.82:0/1216077964 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf5659c700 1 -- 172.21.15.82:0/1216077964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c009550 con 0x7faf58105f40 2023-07-14T19:20:53.192 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf5659c700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf581990a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009fb0 tx=0x7faf4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:53.193 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf4c003930 con 0x7faf58105f40 2023-07-14T19:20:53.193 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf4c003a90 con 0x7faf58105f40 2023-07-14T19:20:53.193 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf58199840 con 0x7faf58105f40 2023-07-14T19:20:53.193 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf4c0195e0 con 0x7faf58105f40 2023-07-14T19:20:53.194 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.541+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf58199cc0 con 0x7faf58105f40 2023-07-14T19:20:53.194 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.542+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7faf4c019850 con 0x7faf58105f40 2023-07-14T19:20:53.194 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.542+0000 7faf477fe700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7faf40038710 0x7faf4003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:53.194 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.542+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faf4c04c760 con 0x7faf58105f40 2023-07-14T19:20:53.195 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.542+0000 7faf55d9b700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7faf40038710 0x7faf4003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:53.195 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.542+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf58109460 con 0x7faf58105f40 2023-07-14T19:20:53.195 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.542+0000 7faf55d9b700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7faf40038710 0x7faf4003abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf48006fd0 tx=0x7faf48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:53.195 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.547+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7faf4c019b50 con 0x7faf58105f40 2023-07-14T19:20:53.196 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.759+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faf58100e90 con 0x7faf40038710 2023-07-14T19:20:53.197 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.764+0000 7faf477fe700 1 -- 172.21.15.82:0/1216077964 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faf58100e90 con 0x7faf40038710 2023-07-14T19:20:53.197 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.766+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7faf40038710 msgr2=0x7faf4003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:53.197 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.766+0000 7faf5cdd3700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7faf40038710 0x7faf4003abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf48006fd0 tx=0x7faf48006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:53.197 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 msgr2=0x7faf581990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:53.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf581990a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009fb0 tx=0x7faf4c011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:53.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 shutdown_connections 2023-07-14T19:20:53.198 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7faf40038710 0x7faf4003abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:53.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 --2- 172.21.15.82:0/1216077964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf58105f40 0x7faf581990a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:53.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 >> 172.21.15.82:0/1216077964 conn(0x7faf580ff9a0 msgr2=0x7faf58100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:53.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 shutdown_connections 2023-07-14T19:20:53.199 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:52.767+0000 7faf5cdd3700 1 -- 172.21.15.82:0/1216077964 wait complete. 2023-07-14T19:20:53.199 INFO:teuthology.orchestra.run.smithi082.stdout:Deploying unmanaged mgr service... 2023-07-14T19:20:54.168 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:53 smithi082 ceph-mon[127461]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:54.169 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:53 smithi082 ceph-mon[127461]: Saving service mon spec with placement count:5 2023-07-14T19:20:54.169 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:53 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:54.336 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-14T19:20:54.336 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.689+0000 7f2e5cf4d700 1 Processor -- start 2023-07-14T19:20:54.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.689+0000 7f2e5cf4d700 1 -- start start 2023-07-14T19:20:54.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.689+0000 7f2e5cf4d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:54.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.689+0000 7f2e5cf4d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58104c40 con 0x7f2e581042f0 2023-07-14T19:20:54.337 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:54.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39614/0 (socket says 172.21.15.82:39614) 2023-07-14T19:20:54.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5659c700 1 -- 172.21.15.82:0/4022598595 learned_addr learned my addr 172.21.15.82:0/4022598595 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:54.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5659c700 1 -- 172.21.15.82:0/4022598595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e58104d80 con 0x7f2e581042f0 2023-07-14T19:20:54.338 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5659c700 1 --2- 172.21.15.82:0/4022598595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58104700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e480098a0 tx=0x7f2e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c267d260d2ad894b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:54.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5559a700 1 -- 172.21.15.82:0/4022598595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e48004160 con 0x7f2e581042f0 2023-07-14T19:20:54.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5559a700 1 -- 172.21.15.82:0/4022598595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e480042c0 con 0x7f2e581042f0 2023-07-14T19:20:54.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.690+0000 7f2e5559a700 1 -- 172.21.15.82:0/4022598595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e48011a00 con 0x7f2e581042f0 2023-07-14T19:20:54.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/4022598595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 msgr2=0x7f2e58104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:54.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 --2- 172.21.15.82:0/4022598595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58104700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2e480098a0 tx=0x7f2e48009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:54.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/4022598595 shutdown_connections 2023-07-14T19:20:54.340 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 --2- 172.21.15.82:0/4022598595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58104700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:54.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/4022598595 >> 172.21.15.82:0/4022598595 conn(0x7f2e580ff9a0 msgr2=0x7f2e58101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:54.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/4022598595 shutdown_connections 2023-07-14T19:20:54.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.691+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/4022598595 wait complete. 2023-07-14T19:20:54.341 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5cf4d700 1 Processor -- start 2023-07-14T19:20:54.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5cf4d700 1 -- start start 2023-07-14T19:20:54.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5cf4d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58079b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:54.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5cf4d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5819bed0 con 0x7f2e581042f0 2023-07-14T19:20:54.342 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58079b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:54.343 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58079b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39616/0 (socket says 172.21.15.82:39616) 2023-07-14T19:20:54.343 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5659c700 1 -- 172.21.15.82:0/315735335 learned_addr learned my addr 172.21.15.82:0/315735335 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:54.343 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5659c700 1 -- 172.21.15.82:0/315735335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48009550 con 0x7f2e581042f0 2023-07-14T19:20:54.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.692+0000 7f2e5659c700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58079b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009130 tx=0x7f2e48011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:54.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.693+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e48003b70 con 0x7f2e581042f0 2023-07-14T19:20:54.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.693+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e48003cd0 con 0x7f2e581042f0 2023-07-14T19:20:54.344 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.693+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5819c010 con 0x7f2e581042f0 2023-07-14T19:20:54.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.693+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e480197d0 con 0x7f2e581042f0 2023-07-14T19:20:54.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.693+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5819c430 con 0x7f2e581042f0 2023-07-14T19:20:54.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.693+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2e48019a10 con 0x7f2e581042f0 2023-07-14T19:20:54.345 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.694+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58061c70 con 0x7f2e581042f0 2023-07-14T19:20:54.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.694+0000 7f2e477fe700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f2e40038710 0x7f2e4003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:54.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.694+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2e4804c930 con 0x7f2e581042f0 2023-07-14T19:20:54.346 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.694+0000 7f2e55d9b700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f2e40038710 0x7f2e4003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:54.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.694+0000 7f2e55d9b700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f2e40038710 0x7f2e4003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c006fd0 tx=0x7f2e4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:54.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.698+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2e48018db0 con 0x7f2e581042f0 2023-07-14T19:20:54.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.900+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2e58101cd0 con 0x7f2e40038710 2023-07-14T19:20:54.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.905+0000 7f2e477fe700 1 -- 172.21.15.82:0/315735335 <== mgr.14120 v2:172.21.15.82:6800/1035467551 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2e58101cd0 con 0x7f2e40038710 2023-07-14T19:20:54.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f2e40038710 msgr2=0x7f2e4003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:54.347 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f2e40038710 0x7f2e4003abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c006fd0 tx=0x7f2e4c006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:54.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 msgr2=0x7f2e58079b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:54.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58079b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009130 tx=0x7f2e48011ad0 comp rx=0 tx=0).stop 2023-07-14T19:20:54.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 shutdown_connections 2023-07-14T19:20:54.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f2e40038710 0x7f2e4003abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:54.348 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 --2- 172.21.15.82:0/315735335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e581042f0 0x7f2e58079b10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:54.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 >> 172.21.15.82:0/315735335 conn(0x7f2e580ff9a0 msgr2=0x7f2e581010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:54.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 shutdown_connections 2023-07-14T19:20:54.349 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:53.907+0000 7f2e5cf4d700 1 -- 172.21.15.82:0/315735335 wait complete. 2023-07-14T19:20:55.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:54 smithi082 ceph-mon[127461]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:20:55.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:54 smithi082 ceph-mon[127461]: Saving service mgr spec with placement count:2 2023-07-14T19:20:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:54 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:54 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:54 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:55.273 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.764+0000 7fdce045b700 1 Processor -- start 2023-07-14T19:20:55.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.764+0000 7fdce045b700 1 -- start start 2023-07-14T19:20:55.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.764+0000 7fdce045b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:55.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.764+0000 7fdce045b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd8104c40 con 0x7fdcd81042f0 2023-07-14T19:20:55.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcde1f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:55.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcde1f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39620/0 (socket says 172.21.15.82:39620) 2023-07-14T19:20:55.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcde1f7700 1 -- 172.21.15.82:0/3691747922 learned_addr learned my addr 172.21.15.82:0/3691747922 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcde1f7700 1 -- 172.21.15.82:0/3691747922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8104d80 con 0x7fdcd81042f0 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcde1f7700 1 --2- 172.21.15.82:0/3691747922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8104700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40098a0 tx=0x7fdcd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89265c81c66f5a28 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcdd1f5700 1 -- 172.21.15.82:0/3691747922 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd4004160 con 0x7fdcd81042f0 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcdd1f5700 1 -- 172.21.15.82:0/3691747922 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcd40042c0 con 0x7fdcd81042f0 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.765+0000 7fdcdd1f5700 1 -- 172.21.15.82:0/3691747922 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd40119b0 con 0x7fdcd81042f0 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 -- 172.21.15.82:0/3691747922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 msgr2=0x7fdcd8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 --2- 172.21.15.82:0/3691747922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8104700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40098a0 tx=0x7fdcd4009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:55.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 -- 172.21.15.82:0/3691747922 shutdown_connections 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 --2- 172.21.15.82:0/3691747922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8104700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 -- 172.21.15.82:0/3691747922 >> 172.21.15.82:0/3691747922 conn(0x7fdcd80ff9a0 msgr2=0x7fdcd8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 -- 172.21.15.82:0/3691747922 shutdown_connections 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.766+0000 7fdce045b700 1 -- 172.21.15.82:0/3691747922 wait complete. 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdce045b700 1 Processor -- start 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdce045b700 1 -- start start 2023-07-14T19:20:55.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdce045b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8079b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdce045b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd819bdd0 con 0x7fdcd81042f0 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdcde1f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8079b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdcde1f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8079b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39622/0 (socket says 172.21.15.82:39622) 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdcde1f7700 1 -- 172.21.15.82:0/3862976116 learned_addr learned my addr 172.21.15.82:0/3862976116 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdcde1f7700 1 -- 172.21.15.82:0/3862976116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009550 con 0x7fdcd81042f0 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdcde1f7700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8079b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4003e60 tx=0x7fdcd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd4003970 con 0x7fdcd81042f0 2023-07-14T19:20:55.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.767+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcd4003ad0 con 0x7fdcd81042f0 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.768+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd819bf10 con 0x7fdcd81042f0 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.768+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd40195e0 con 0x7fdcd81042f0 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.768+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd819c330 con 0x7fdcd81042f0 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.768+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fdcd4019850 con 0x7fdcd81042f0 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.768+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd8061c70 con 0x7fdcd81042f0 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.768+0000 7fdccf7fe700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7fdcc4038710 0x7fdcc403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:55.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.769+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdcd404c830 con 0x7fdcd81042f0 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.769+0000 7fdcdd9f6700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7fdcc4038710 0x7fdcc403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.769+0000 7fdcdd9f6700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7fdcc4038710 0x7fdcc403abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.773+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fdcd402b340 con 0x7fdcd81042f0 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.979+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fdcd81061f0 con 0x7fdcd81042f0 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.985+0000 7fdccf7fe700 1 -- 172.21.15.82:0/3862976116 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fdcd4018b40 con 0x7fdcd81042f0 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7fdcc4038710 msgr2=0x7fdcc403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7fdcc4038710 0x7fdcc403abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:55.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 msgr2=0x7fdcd8079b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8079b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4003e60 tx=0x7fdcd4011aa0 comp rx=0 tx=0).stop 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 shutdown_connections 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7fdcc4038710 0x7fdcc403abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 --2- 172.21.15.82:0/3862976116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd81042f0 0x7fdcd8079b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 >> 172.21.15.82:0/3862976116 conn(0x7fdcd80ff9a0 msgr2=0x7fdcd8100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.987+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 shutdown_connections 2023-07-14T19:20:55.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:54.988+0000 7fdce045b700 1 -- 172.21.15.82:0/3862976116 wait complete. 2023-07-14T19:20:56.359 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3862976116' entity='client.admin' 2023-07-14T19:20:56.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47bd1cd700 1 Processor -- start 2023-07-14T19:20:56.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47bd1cd700 1 -- start start 2023-07-14T19:20:56.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47bd1cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b8072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:56.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47bd1cd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80709b0 con 0x7f47b8071e70 2023-07-14T19:20:56.418 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b8072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b8072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39626/0 (socket says 172.21.15.82:39626) 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.768+0000 7f47b6d9d700 1 -- 172.21.15.82:0/2350954226 learned_addr learned my addr 172.21.15.82:0/2350954226 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.769+0000 7f47b6d9d700 1 -- 172.21.15.82:0/2350954226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b810fd90 con 0x7f47b8071e70 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.769+0000 7f47b6d9d700 1 --2- 172.21.15.82:0/2350954226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b8072280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47a80098a0 tx=0x7f47a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4e6171608923b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.769+0000 7f47b5d9b700 1 -- 172.21.15.82:0/2350954226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47a8004160 con 0x7f47b8071e70 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.769+0000 7f47b5d9b700 1 -- 172.21.15.82:0/2350954226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a80042c0 con 0x7f47b8071e70 2023-07-14T19:20:56.419 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.769+0000 7f47b5d9b700 1 -- 172.21.15.82:0/2350954226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47a80119b0 con 0x7f47b8071e70 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2350954226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 msgr2=0x7f47b8072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 --2- 172.21.15.82:0/2350954226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b8072280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47a80098a0 tx=0x7f47a8009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2350954226 shutdown_connections 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 --2- 172.21.15.82:0/2350954226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b8072280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2350954226 >> 172.21.15.82:0/2350954226 conn(0x7f47b806bfc0 msgr2=0x7f47b806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2350954226 shutdown_connections 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.770+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2350954226 wait complete. 2023-07-14T19:20:56.420 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47bd1cd700 1 Processor -- start 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47bd1cd700 1 -- start start 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47bd1cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b81a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47bd1cd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b81a0d70 con 0x7f47b8071e70 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b81a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47b6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b81a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39628/0 (socket says 172.21.15.82:39628) 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.771+0000 7f47b6d9d700 1 -- 172.21.15.82:0/2390724205 learned_addr learned my addr 172.21.15.82:0/2390724205 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f47b6d9d700 1 -- 172.21.15.82:0/2390724205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8009550 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f47b6d9d700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b81a0830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47a8003f00 tx=0x7f47a8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47a8003b20 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a8003c80 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b81a0fd0 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47a8019710 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.772+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b81a1450 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.773+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f47a8019980 con 0x7f47b8071e70 2023-07-14T19:20:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.773+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8110bd0 con 0x7f47b8071e70 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.773+0000 7f479ffff700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f47a00387b0 0x7f47a003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.773+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f47a804c950 con 0x7f47b8071e70 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.774+0000 7f47b659c700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f47a00387b0 0x7f47a003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.774+0000 7f47b659c700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f47a00387b0 0x7f47a003ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f47ac006fd0 tx=0x7f47ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.779+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f47a8018db0 con 0x7f47b8071e70 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.981+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f47b81a41f0 con 0x7f47b8071e70 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.984+0000 7f479ffff700 1 -- 172.21.15.82:0/2390724205 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f47a8018350 con 0x7f47b8071e70 2023-07-14T19:20:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.986+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f47a00387b0 msgr2=0x7f47a003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.986+0000 7f47bd1cd700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f47a00387b0 0x7f47a003ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f47ac006fd0 tx=0x7f47ac006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.986+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 msgr2=0x7f47b81a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.986+0000 7f47bd1cd700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b81a0830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47a8003f00 tx=0x7f47a8011a30 comp rx=0 tx=0).stop 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.987+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 shutdown_connections 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.987+0000 7f47bd1cd700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f47a00387b0 0x7f47a003ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.987+0000 7f47bd1cd700 1 --2- 172.21.15.82:0/2390724205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8071e70 0x7f47b81a0830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:56.424 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.987+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 >> 172.21.15.82:0/2390724205 conn(0x7f47b806bfc0 msgr2=0x7f47b806d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:56.425 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.987+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 shutdown_connections 2023-07-14T19:20:56.425 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:55.987+0000 7f47bd1cd700 1 -- 172.21.15.82:0/2390724205 wait complete. 2023-07-14T19:20:56.425 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling the dashboard module... 2023-07-14T19:20:57.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:56 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2390724205' entity='client.admin' 2023-07-14T19:20:57.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:56 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:57.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:56 smithi082 ceph-mon[127461]: from='mgr.14120 172.21.15.82:0/1671678243' entity='mgr.x' 2023-07-14T19:20:58.281 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2550563086' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-14T19:20:58.282 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: ignoring --setuser ceph since I am not root 2023-07-14T19:20:58.282 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: ignoring --setgroup ceph since I am not root 2023-07-14T19:20:58.282 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:58.198+0000 7fd21e7ce200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:20:58.282 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:58.280+0000 7fd21e7ce200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:20:58.638 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:58.372+0000 7fd21e7ce200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:20:58.638 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:58.467+0000 7fd21e7ce200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:20:58.670 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba2f71700 1 Processor -- start 2023-07-14T19:20:58.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba2f71700 1 -- start start 2023-07-14T19:20:58.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba2f71700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:58.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba2f71700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c079d20 con 0x7f8b9c105f40 2023-07-14T19:20:58.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba0d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:58.671 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba0d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39632/0 (socket says 172.21.15.82:39632) 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.875+0000 7f8ba0d0d700 1 -- 172.21.15.82:0/2354026494 learned_addr learned my addr 172.21.15.82:0/2354026494 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.876+0000 7f8ba0d0d700 1 -- 172.21.15.82:0/2354026494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c079e60 con 0x7f8b9c105f40 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.876+0000 7f8ba0d0d700 1 --2- 172.21.15.82:0/2354026494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c108320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b900098a0 tx=0x7f8b90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a680fb5e2200ff1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.876+0000 7f8b9b7fe700 1 -- 172.21.15.82:0/2354026494 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b90004160 con 0x7f8b9c105f40 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.876+0000 7f8b9b7fe700 1 -- 172.21.15.82:0/2354026494 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b900042c0 con 0x7f8b9c105f40 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.876+0000 7f8b9b7fe700 1 -- 172.21.15.82:0/2354026494 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b900119b0 con 0x7f8b9c105f40 2023-07-14T19:20:58.672 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2354026494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 msgr2=0x7f8b9c108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 --2- 172.21.15.82:0/2354026494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c108320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b900098a0 tx=0x7f8b90009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2354026494 shutdown_connections 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 --2- 172.21.15.82:0/2354026494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c108320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2354026494 >> 172.21.15.82:0/2354026494 conn(0x7f8b9c0ff9a0 msgr2=0x7f8b9c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2354026494 shutdown_connections 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2354026494 wait complete. 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 Processor -- start 2023-07-14T19:20:58.673 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- start start 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c1058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.877+0000 7f8ba2f71700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c105de0 con 0x7f8b9c105f40 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba0d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c1058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba0d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c1058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39634/0 (socket says 172.21.15.82:39634) 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba0d0d700 1 -- 172.21.15.82:0/2550563086 learned_addr learned my addr 172.21.15.82:0/2550563086 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba0d0d700 1 -- 172.21.15.82:0/2550563086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b90009550 con 0x7f8b9c105f40 2023-07-14T19:20:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba0d0d700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c1058a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b90003e60 tx=0x7f8b90011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b90003970 con 0x7f8b9c105f40 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b90003ad0 con 0x7f8b9c105f40 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c103f50 con 0x7f8b9c105f40 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b90019590 con 0x7f8b9c105f40 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.878+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c1043d0 con 0x7f8b9c105f40 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.879+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8b90019800 con 0x7f8b9c105f40 2023-07-14T19:20:58.675 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.879+0000 7f8b99ffb700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f8b8c0386f0 0x7f8b8c03aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:58.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.879+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9c1094b0 con 0x7f8b9c105f40 2023-07-14T19:20:58.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.879+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8b9004c7f0 con 0x7f8b9c105f40 2023-07-14T19:20:58.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.879+0000 7f8b9bfff700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f8b8c0386f0 0x7f8b8c03aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:58.676 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.880+0000 7f8b9bfff700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f8b8c0386f0 0x7f8b8c03aba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006fd0 tx=0x7f8b88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:58.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:56.884+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8b90019b00 con 0x7f8b9c105f40 2023-07-14T19:20:58.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:57.147+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8b9c104a30 con 0x7f8b9c105f40 2023-07-14T19:20:58.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.027+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f8b90019ce0 con 0x7f8b9c105f40 2023-07-14T19:20:58.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.028+0000 7f8b99ffb700 1 -- 172.21.15.82:0/2550563086 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8b90019b00 con 0x7f8b9c105f40 2023-07-14T19:20:58.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f8b8c0386f0 msgr2=0x7f8b8c03aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:58.677 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f8b8c0386f0 0x7f8b8c03aba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8b88006fd0 tx=0x7f8b88006e40 comp rx=0 tx=0).stop 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 msgr2=0x7f8b9c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c1058a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b90003e60 tx=0x7f8b90011aa0 comp rx=0 tx=0).stop 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 shutdown_connections 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f8b8c0386f0 0x7f8b8c03aba0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 --2- 172.21.15.82:0/2550563086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b9c105f40 0x7f8b9c1058a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 >> 172.21.15.82:0/2550563086 conn(0x7f8b9c0ff9a0 msgr2=0x7f8b9c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 shutdown_connections 2023-07-14T19:20:58.678 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:58.031+0000 7f8ba2f71700 1 -- 172.21.15.82:0/2550563086 wait complete. 2023-07-14T19:20:58.888 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:58.636+0000 7fd21e7ce200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:20:59.202 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:59 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2550563086' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-14T19:20:59.202 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:20:59 smithi082 ceph-mon[127461]: mgrmap e9: x(active, since 15s) 2023-07-14T19:20:59.202 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:58.967+0000 7fd21e7ce200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:20:59.203 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:59.062+0000 7fd21e7ce200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:20:59.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:59.201+0000 7fd21e7ce200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:20:59.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:59.298+0000 7fd21e7ce200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:20:59.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:20:59.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-14T19:20:59.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-14T19:20:59.719 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.142+0000 7f95fe8cc700 1 Processor -- start 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.142+0000 7f95fe8cc700 1 -- start start 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.142+0000 7f95fe8cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f8108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.142+0000 7f95fe8cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f8079d20 con 0x7f95f8105f40 2023-07-14T19:20:59.720 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.142+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f8108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.142+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f8108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39648/0 (socket says 172.21.15.82:39648) 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.143+0000 7f95f7fff700 1 -- 172.21.15.82:0/3238789415 learned_addr learned my addr 172.21.15.82:0/3238789415 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.143+0000 7f95f7fff700 1 -- 172.21.15.82:0/3238789415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8079e60 con 0x7f95f8105f40 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.143+0000 7f95f7fff700 1 --2- 172.21.15.82:0/3238789415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f8108320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95e80098a0 tx=0x7f95e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c951f0651fb68b30 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.143+0000 7f95f6ffd700 1 -- 172.21.15.82:0/3238789415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e8004160 con 0x7f95f8105f40 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.143+0000 7f95f6ffd700 1 -- 172.21.15.82:0/3238789415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e80042c0 con 0x7f95f8105f40 2023-07-14T19:20:59.721 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.143+0000 7f95f6ffd700 1 -- 172.21.15.82:0/3238789415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e80119b0 con 0x7f95f8105f40 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 -- 172.21.15.82:0/3238789415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 msgr2=0x7f95f8108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 --2- 172.21.15.82:0/3238789415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f8108320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95e80098a0 tx=0x7f95e8009bb0 comp rx=0 tx=0).stop 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 -- 172.21.15.82:0/3238789415 shutdown_connections 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 --2- 172.21.15.82:0/3238789415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f8108320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 -- 172.21.15.82:0/3238789415 >> 172.21.15.82:0/3238789415 conn(0x7f95f80ff9a0 msgr2=0x7f95f8101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 -- 172.21.15.82:0/3238789415 shutdown_connections 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.144+0000 7f95fe8cc700 1 -- 172.21.15.82:0/3238789415 wait complete. 2023-07-14T19:20:59.722 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95fe8cc700 1 Processor -- start 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95fe8cc700 1 -- start start 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95fe8cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f81058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95fe8cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f8105de0 con 0x7f95f8105f40 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f81058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f81058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39650/0 (socket says 172.21.15.82:39650) 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95f7fff700 1 -- 172.21.15.82:0/2954238103 learned_addr learned my addr 172.21.15.82:0/2954238103 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:20:59.723 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.145+0000 7f95f7fff700 1 -- 172.21.15.82:0/2954238103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8009550 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.146+0000 7f95f7fff700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f81058a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95e8003e60 tx=0x7f95e8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.146+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e8003970 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.146+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f8103f50 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.146+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e8003ad0 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.146+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95e8019590 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.146+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f81043d0 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.147+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f95e8019890 con 0x7f95f8105f40 2023-07-14T19:20:59.724 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.147+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f81094b0 con 0x7f95f8105f40 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.147+0000 7f95f57fa700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 0x7f95e003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.147+0000 7f95f77fe700 1 -- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 msgr2=0x7f95e003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.147+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f95e804c840 con 0x7f95f8105f40 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.147+0000 7f95f77fe700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 0x7f95e003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.156+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f95e8018db0 con 0x7f95f8105f40 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.348+0000 7f95f77fe700 1 -- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 msgr2=0x7f95e003aba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:20:59.725 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.348+0000 7f95f77fe700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 0x7f95e003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.401+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f95f8104a30 con 0x7f95f8105f40 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.401+0000 7f95f57fa700 1 -- 172.21.15.82:0/2954238103 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f95e8018350 con 0x7f95f8105f40 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.403+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 msgr2=0x7f95e003aba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.403+0000 7f95fe8cc700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 0x7f95e003aba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.403+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 msgr2=0x7f95f81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.403+0000 7f95fe8cc700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f81058a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f95e8003e60 tx=0x7f95e8011aa0 comp rx=0 tx=0).stop 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.404+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 shutdown_connections 2023-07-14T19:20:59.726 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.404+0000 7f95fe8cc700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7f95e00386f0 0x7f95e003aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:59.727 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.404+0000 7f95fe8cc700 1 --2- 172.21.15.82:0/2954238103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95f8105f40 0x7f95f81058a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:20:59.727 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.404+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 >> 172.21.15.82:0/2954238103 conn(0x7f95f80ff9a0 msgr2=0x7f95f8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:20:59.727 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.404+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 shutdown_connections 2023-07-14T19:20:59.727 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:20:59.404+0000 7f95fe8cc700 1 -- 172.21.15.82:0/2954238103 wait complete. 2023-07-14T19:20:59.727 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for the mgr to restart... 2023-07-14T19:20:59.727 INFO:teuthology.orchestra.run.smithi082.stdout:Waiting for mgr epoch 9... 2023-07-14T19:20:59.915 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:20:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:20:59.733+0000 7fd21e7ce200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:21:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:00 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2954238103' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-14T19:21:00.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:00.320+0000 7fd21e7ce200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:21:00.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:00.489+0000 7fd21e7ce200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:21:01.516 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:01.248+0000 7fd21e7ce200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:21:01.516 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:01.339+0000 7fd21e7ce200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:21:01.802 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:01.515+0000 7fd21e7ce200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:21:02.180 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:01.801+0000 7fd21e7ce200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:21:02.180 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:02.031+0000 7fd21e7ce200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:21:02.180 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:02.178+0000 7fd21e7ce200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:21:02.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:02.440+0000 7fd21e7ce200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:21:03.451 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:03.131+0000 7fd21e7ce200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:21:03.451 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:03.260+0000 7fd21e7ce200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:21:03.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:03.450+0000 7fd21e7ce200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:21:03.715 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:03.563+0000 7fd21e7ce200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:21:04.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:03.778+0000 7fd21e7ce200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:21:04.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:21:04.630+0000 7fd21e7ce200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:21:04.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: Active manager daemon x restarted 2023-07-14T19:21:04.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: Activating manager daemon x 2023-07-14T19:21:04.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: osdmap e3: 0 total, 0 up, 0 in 2023-07-14T19:21:04.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: mgrmap e10: x(active, starting, since 0.00366138s) 2023-07-14T19:21:04.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:04.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T19:21:04.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T19:21:04.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T19:21:04.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T19:21:04.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: Manager daemon x is now available 2023-07-14T19:21:04.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:04.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:05.951 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T19:21:05.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T19:21:05.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: [14/Jul/2023:19:21:04] ENGINE Bus STARTING 2023-07-14T19:21:05.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: [14/Jul/2023:19:21:04] ENGINE Serving on https://172.21.15.82:7150 2023-07-14T19:21:05.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: [14/Jul/2023:19:21:05] ENGINE Serving on http://172.21.15.82:8765 2023-07-14T19:21:05.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: [14/Jul/2023:19:21:05] ENGINE Bus STARTED 2023-07-14T19:21:05.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:05 smithi082 ceph-mon[127461]: mgrmap e11: x(active, since 1.00561s) 2023-07-14T19:21:05.978 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout { 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout } 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.221+0000 7efe110bc700 1 Processor -- start 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe110bc700 1 -- start start 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe110bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c1082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:05.979 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe110bc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c079d60 con 0x7efe0c105f00 2023-07-14T19:21:05.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe0ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c1082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:05.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe0ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c1082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39658/0 (socket says 172.21.15.82:39658) 2023-07-14T19:21:05.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe0ad9d700 1 -- 172.21.15.82:0/3617919103 learned_addr learned my addr 172.21.15.82:0/3617919103 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:05.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe0ad9d700 1 -- 172.21.15.82:0/3617919103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c079ea0 con 0x7efe0c105f00 2023-07-14T19:21:05.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.222+0000 7efe0ad9d700 1 --2- 172.21.15.82:0/3617919103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c1082e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0098a0 tx=0x7efdfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4814e3ad89c90 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:05.980 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe09d9b700 1 -- 172.21.15.82:0/3617919103 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdfc004160 con 0x7efe0c105f00 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe09d9b700 1 -- 172.21.15.82:0/3617919103 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdfc0042c0 con 0x7efe0c105f00 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe09d9b700 1 -- 172.21.15.82:0/3617919103 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdfc0119b0 con 0x7efe0c105f00 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 -- 172.21.15.82:0/3617919103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 msgr2=0x7efe0c1082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 --2- 172.21.15.82:0/3617919103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c1082e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0098a0 tx=0x7efdfc009bb0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 -- 172.21.15.82:0/3617919103 shutdown_connections 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 --2- 172.21.15.82:0/3617919103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c1082e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.981 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 -- 172.21.15.82:0/3617919103 >> 172.21.15.82:0/3617919103 conn(0x7efe0c0ff9a0 msgr2=0x7efe0c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 -- 172.21.15.82:0/3617919103 shutdown_connections 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.223+0000 7efe110bc700 1 -- 172.21.15.82:0/3617919103 wait complete. 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.224+0000 7efe110bc700 1 Processor -- start 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.224+0000 7efe110bc700 1 -- start start 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.224+0000 7efe110bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.224+0000 7efe110bc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c19b850 con 0x7efe0c105f00 2023-07-14T19:21:05.982 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.224+0000 7efe0ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.224+0000 7efe0ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39660/0 (socket says 172.21.15.82:39660) 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efe0ad9d700 1 -- 172.21.15.82:0/2070708268 learned_addr learned my addr 172.21.15.82:0/2070708268 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efe0ad9d700 1 -- 172.21.15.82:0/2070708268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc009550 con 0x7efe0c105f00 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efe0ad9d700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c19b310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdfc003e60 tx=0x7efdfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdfc003970 con 0x7efe0c105f00 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdfc003ad0 con 0x7efe0c105f00 2023-07-14T19:21:05.983 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c19ba50 con 0x7efe0c105f00 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdfc0195e0 con 0x7efe0c105f00 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.225+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c19be70 con 0x7efe0c105f00 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.226+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7efdfc0198a0 con 0x7efe0c105f00 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.226+0000 7efdf3fff700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.226+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 --> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdf403b2d0 con 0x7efdf4038710 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.226+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efdfc04c8c0 con 0x7efe0c105f00 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.226+0000 7efe0a59c700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 msgr2=0x7efdf403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:21:05.984 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.226+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:21:05.985 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.427+0000 7efe0a59c700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 msgr2=0x7efdf403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:21:05.985 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.427+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:21:05.985 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.828+0000 7efe0a59c700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 msgr2=0x7efdf403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:21:05.985 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:00.828+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T19:21:05.985 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:01.629+0000 7efe0a59c700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 msgr2=0x7efdf403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:21:05.985 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:01.629+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:03.230+0000 7efe0a59c700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 msgr2=0x7efdf403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/1035467551 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:03.230+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:04.646+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7efdfc02da00 con 0x7efe0c105f00 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:04.646+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 msgr2=0x7efdf403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:04.646+0000 7efdf3fff700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.647+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7efdfc01ab10 con 0x7efe0c105f00 2023-07-14T19:21:05.986 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.648+0000 7efdf3fff700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdf403bff0 0x7efdf403e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.648+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdf403b2d0 con 0x7efdf403bff0 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.648+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdf403bff0 0x7efdf403e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.648+0000 7efe0a59c700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdf403bff0 0x7efdf403e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe00003a10 tx=0x7efe000092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.649+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7efdf403b2d0 con 0x7efdf403bff0 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.655+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7efe0c19c650 con 0x7efdf403bff0 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efdf3fff700 1 -- 172.21.15.82:0/2070708268 <== mgr.14152 v2:172.21.15.82:6800/504215117 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7efe0c19c650 con 0x7efdf403bff0 2023-07-14T19:21:05.987 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdf403bff0 msgr2=0x7efdf403e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:05.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdf403bff0 0x7efdf403e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe00003a10 tx=0x7efe000092b0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 msgr2=0x7efe0c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:05.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c19b310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efdfc003e60 tx=0x7efdfc011aa0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 shutdown_connections 2023-07-14T19:21:05.988 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdf403bff0 0x7efdf403e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe0c105f00 0x7efe0c19b310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 --2- 172.21.15.82:0/2070708268 >> [v2:172.21.15.82:6800/1035467551,v1:172.21.15.82:6801/1035467551] conn(0x7efdf4038710 0x7efdf403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:05.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 >> 172.21.15.82:0/2070708268 conn(0x7efe0c0ff9a0 msgr2=0x7efe0c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:05.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 shutdown_connections 2023-07-14T19:21:05.989 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:05.656+0000 7efe110bc700 1 -- 172.21.15.82:0/2070708268 wait complete. 2023-07-14T19:21:05.990 INFO:teuthology.orchestra.run.smithi082.stdout:mgr epoch 9 is available 2023-07-14T19:21:05.990 INFO:teuthology.orchestra.run.smithi082.stdout:Generating a dashboard self-signed certificate... 2023-07-14T19:21:06.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:06 smithi082 ceph-mon[127461]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-14T19:21:06.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:06 smithi082 ceph-mon[127461]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-14T19:21:06.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:07.476 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-14T19:21:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.767+0000 7fc5dc280700 1 Processor -- start 2023-07-14T19:21:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.767+0000 7fc5dc280700 1 -- start start 2023-07-14T19:21:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.767+0000 7fc5dc280700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4105ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.767+0000 7fc5dc280700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4106220 con 0x7fc5d41058d0 2023-07-14T19:21:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5da01c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4105ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5da01c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4105ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39714/0 (socket says 172.21.15.82:39714) 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5da01c700 1 -- 172.21.15.82:0/3290413078 learned_addr learned my addr 172.21.15.82:0/3290413078 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5da01c700 1 -- 172.21.15.82:0/3290413078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4106360 con 0x7fc5d41058d0 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5da01c700 1 --2- 172.21.15.82:0/3290413078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4105ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009e80 tx=0x7fc5cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f86feeb1179491a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5d901a700 1 -- 172.21.15.82:0/3290413078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc00e070 con 0x7fc5d41058d0 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5d901a700 1 -- 172.21.15.82:0/3290413078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5cc00b8c0 con 0x7fc5d41058d0 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.768+0000 7fc5d901a700 1 -- 172.21.15.82:0/3290413078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc0043e0 con 0x7fc5d41058d0 2023-07-14T19:21:07.478 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 -- 172.21.15.82:0/3290413078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 msgr2=0x7fc5d4105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:07.479 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 --2- 172.21.15.82:0/3290413078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4105ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009e80 tx=0x7fc5cc00b330 comp rx=0 tx=0).stop 2023-07-14T19:21:07.479 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 -- 172.21.15.82:0/3290413078 shutdown_connections 2023-07-14T19:21:07.479 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 --2- 172.21.15.82:0/3290413078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4105ce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:07.479 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 -- 172.21.15.82:0/3290413078 >> 172.21.15.82:0/3290413078 conn(0x7fc5d40ff120 msgr2=0x7fc5d4101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:07.479 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 -- 172.21.15.82:0/3290413078 shutdown_connections 2023-07-14T19:21:07.479 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.769+0000 7fc5dc280700 1 -- 172.21.15.82:0/3290413078 wait complete. 2023-07-14T19:21:07.480 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5dc280700 1 Processor -- start 2023-07-14T19:21:07.480 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5dc280700 1 -- start start 2023-07-14T19:21:07.480 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5dc280700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:07.480 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5dc280700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41984c0 con 0x7fc5d41058d0 2023-07-14T19:21:07.481 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5da01c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:07.481 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5da01c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39716/0 (socket says 172.21.15.82:39716) 2023-07-14T19:21:07.481 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.770+0000 7fc5da01c700 1 -- 172.21.15.82:0/2960234249 learned_addr learned my addr 172.21.15.82:0/2960234249 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:07.481 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5da01c700 1 -- 172.21.15.82:0/2960234249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc009b30 con 0x7fc5d41058d0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5da01c700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4197f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009e50 tx=0x7fc5cc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc00e040 con 0x7fc5d41058d0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5cc00be30 con 0x7fc5d41058d0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d41986c0 con 0x7fc5d41058d0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc01c750 con 0x7fc5d41058d0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.771+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d4198b40 con 0x7fc5d41058d0 2023-07-14T19:21:07.482 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.772+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fc5cc01c990 con 0x7fc5d41058d0 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.772+0000 7fc5cb7fe700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc5c00385f0 0x7fc5c003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.772+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc5cc04c5d0 con 0x7fc5d41058d0 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.772+0000 7fc5d981b700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc5c00385f0 0x7fc5c003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.772+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d4061c70 con 0x7fc5d41058d0 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.772+0000 7fc5d981b700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc5c00385f0 0x7fc5c003aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006fd0 tx=0x7fc5c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.776+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc5cc020080 con 0x7fc5d41058d0 2023-07-14T19:21:07.483 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:06.988+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5d4100540 con 0x7fc5c00385f0 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.055+0000 7fc5cb7fe700 1 -- 172.21.15.82:0/2960234249 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc5d4100540 con 0x7fc5c00385f0 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc5c00385f0 msgr2=0x7fc5c003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc5c00385f0 0x7fc5c003aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4006fd0 tx=0x7fc5c4006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 msgr2=0x7fc5d4197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4197f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009e50 tx=0x7fc5cc003d00 comp rx=0 tx=0).stop 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 shutdown_connections 2023-07-14T19:21:07.484 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc5c00385f0 0x7fc5c003aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:07.485 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 --2- 172.21.15.82:0/2960234249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d41058d0 0x7fc5d4197f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:07.485 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.057+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 >> 172.21.15.82:0/2960234249 conn(0x7fc5d40ff120 msgr2=0x7fc5d40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:07.485 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.058+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 shutdown_connections 2023-07-14T19:21:07.485 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.058+0000 7fc5dc280700 1 -- 172.21.15.82:0/2960234249 wait complete. 2023-07-14T19:21:07.485 INFO:teuthology.orchestra.run.smithi082.stdout:Creating initial admin user... 2023-07-14T19:21:08.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:08 smithi082 ceph-mon[127461]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:08.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:08.324 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:08.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:08.325 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:08 smithi082 ceph-mon[127461]: mgrmap e12: x(active, since 2s) 2023-07-14T19:21:09.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$sSN42Tcu8F1OwCnaVXixruuglY1cXuzBG./N5vksWOk.jgm.nIUKu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689362468, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-14T19:21:09.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.966+0000 7f1c5e4ae700 1 Processor -- start 2023-07-14T19:21:09.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.966+0000 7f1c5e4ae700 1 -- start start 2023-07-14T19:21:09.022 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.966+0000 7f1c5e4ae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:09.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.966+0000 7f1c5e4ae700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c58104c40 con 0x7f1c581042f0 2023-07-14T19:21:09.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:09.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39720/0 (socket says 172.21.15.82:39720) 2023-07-14T19:21:09.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c57fff700 1 -- 172.21.15.82:0/1030181646 learned_addr learned my addr 172.21.15.82:0/1030181646 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:09.023 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c57fff700 1 -- 172.21.15.82:0/1030181646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58104d80 con 0x7f1c581042f0 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c57fff700 1 --2- 172.21.15.82:0/1030181646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58104700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c480098a0 tx=0x7f1c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982c50a1aa149c77 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c56ffd700 1 -- 172.21.15.82:0/1030181646 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c48004160 con 0x7f1c581042f0 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c56ffd700 1 -- 172.21.15.82:0/1030181646 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c480042c0 con 0x7f1c581042f0 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.967+0000 7f1c56ffd700 1 -- 172.21.15.82:0/1030181646 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c480119b0 con 0x7f1c581042f0 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/1030181646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 msgr2=0x7f1c58104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 --2- 172.21.15.82:0/1030181646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58104700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c480098a0 tx=0x7f1c48009bb0 comp rx=0 tx=0).stop 2023-07-14T19:21:09.024 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/1030181646 shutdown_connections 2023-07-14T19:21:09.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 --2- 172.21.15.82:0/1030181646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58104700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:09.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/1030181646 >> 172.21.15.82:0/1030181646 conn(0x7f1c580ff9a0 msgr2=0x7f1c58101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:09.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/1030181646 shutdown_connections 2023-07-14T19:21:09.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.968+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/1030181646 wait complete. 2023-07-14T19:21:09.025 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c5e4ae700 1 Processor -- start 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c5e4ae700 1 -- start start 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c5e4ae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58079b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c5e4ae700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5819be10 con 0x7f1c581042f0 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58079b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c57fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58079b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39722/0 (socket says 172.21.15.82:39722) 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.969+0000 7f1c57fff700 1 -- 172.21.15.82:0/597910341 learned_addr learned my addr 172.21.15.82:0/597910341 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:09.026 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c57fff700 1 -- 172.21.15.82:0/597910341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c48009550 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c57fff700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58079b30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1c48011980 tx=0x7f1c48011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c48003a60 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/597910341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5819bf50 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c48003bc0 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c4801a670 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.970+0000 7f1c5e4ae700 1 -- 172.21.15.82:0/597910341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5819c370 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.971+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1c4801a930 con 0x7f1c581042f0 2023-07-14T19:21:09.027 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.971+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c380052f0 con 0x7f1c581042f0 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.971+0000 7f1c557fa700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c40038760 0x7f1c4003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.971+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c4804c9d0 con 0x7f1c581042f0 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.972+0000 7f1c577fe700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c40038760 0x7f1c4003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.972+0000 7f1c577fe700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c40038760 0x7f1c4003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c006fd0 tx=0x7f1c4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:07.976+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1c480233a0 con 0x7f1c581042f0 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.197+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1c38002400 con 0x7f1c40038760 2023-07-14T19:21:09.028 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.424+0000 7f1c557fa700 1 -- 172.21.15.82:0/597910341 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1c38002400 con 0x7f1c40038760 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c40038760 msgr2=0x7f1c4003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c40038760 0x7f1c4003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c006fd0 tx=0x7f1c4c006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 msgr2=0x7f1c58079b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58079b30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1c48011980 tx=0x7f1c48011a60 comp rx=0 tx=0).stop 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 shutdown_connections 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c40038760 0x7f1c4003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.426+0000 7f1c3effd700 1 --2- 172.21.15.82:0/597910341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c581042f0 0x7f1c58079b30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:09.029 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.427+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 >> 172.21.15.82:0/597910341 conn(0x7f1c580ff9a0 msgr2=0x7f1c58100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:09.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.427+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 shutdown_connections 2023-07-14T19:21:09.030 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:08.427+0000 7f1c3effd700 1 -- 172.21.15.82:0/597910341 wait complete. 2023-07-14T19:21:09.030 INFO:teuthology.orchestra.run.smithi082.stdout:Fetching dashboard port number... 2023-07-14T19:21:09.685 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:09 smithi082 ceph-mon[127461]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:09.685 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:10.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stdout 8443 2023-07-14T19:21:10.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1dae7a700 1 Processor -- start 2023-07-14T19:21:10.065 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1dae7a700 1 -- start start 2023-07-14T19:21:10.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1dae7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d4103e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:10.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1dae7a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d41043c0 con 0x7fd1d4103a70 2023-07-14T19:21:10.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1d8c16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d4103e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:10.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1d8c16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d4103e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39726/0 (socket says 172.21.15.82:39726) 2023-07-14T19:21:10.066 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.465+0000 7fd1d8c16700 1 -- 172.21.15.82:0/2807525810 learned_addr learned my addr 172.21.15.82:0/2807525810 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1d8c16700 1 -- 172.21.15.82:0/2807525810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d4104500 con 0x7fd1d4103a70 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1d8c16700 1 --2- 172.21.15.82:0/2807525810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d4103e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800aa40 tx=0x7fd1c800ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75be14c2cd938a63 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1d37fe700 1 -- 172.21.15.82:0/2807525810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8004160 con 0x7fd1d4103a70 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1d37fe700 1 -- 172.21.15.82:0/2807525810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c800cdb0 con 0x7fd1d4103a70 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1d37fe700 1 -- 172.21.15.82:0/2807525810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8003c30 con 0x7fd1d4103a70 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1dae7a700 1 -- 172.21.15.82:0/2807525810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 msgr2=0x7fd1d4103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:10.067 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1dae7a700 1 --2- 172.21.15.82:0/2807525810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d4103e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800aa40 tx=0x7fd1c800ad50 comp rx=0 tx=0).stop 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1dae7a700 1 -- 172.21.15.82:0/2807525810 shutdown_connections 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1dae7a700 1 --2- 172.21.15.82:0/2807525810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d4103e80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.466+0000 7fd1dae7a700 1 -- 172.21.15.82:0/2807525810 >> 172.21.15.82:0/2807525810 conn(0x7fd1d40ff120 msgr2=0x7fd1d4101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1dae7a700 1 -- 172.21.15.82:0/2807525810 shutdown_connections 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1dae7a700 1 -- 172.21.15.82:0/2807525810 wait complete. 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1dae7a700 1 Processor -- start 2023-07-14T19:21:10.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1dae7a700 1 -- start start 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1dae7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d418c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1dae7a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d418ccf0 con 0x7fd1d4103a70 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1d8c16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d418c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1d8c16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d418c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39728/0 (socket says 172.21.15.82:39728) 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.467+0000 7fd1d8c16700 1 -- 172.21.15.82:0/1344311338 learned_addr learned my addr 172.21.15.82:0/1344311338 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1d8c16700 1 -- 172.21.15.82:0/1344311338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c800a6f0 con 0x7fd1d4103a70 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1d8c16700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d418c7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800a240 tx=0x7fd1c80041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:10.069 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8010600 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c8010c20 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d418ae00 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8021c20 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.468+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d418b220 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.469+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd1c802b440 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.469+0000 7fd1d1ffb700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd1c4038760 0x7fd1c403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.469+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d418bc30 con 0x7fd1d4103a70 2023-07-14T19:21:10.070 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.469+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1c804ccf0 con 0x7fd1d4103a70 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.469+0000 7fd1d3fff700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd1c4038760 0x7fd1c403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.470+0000 7fd1d3fff700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd1c4038760 0x7fd1c403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0006fd0 tx=0x7fd1c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.474+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd1c802be40 con 0x7fd1d4103a70 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.659+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd1d4192300 con 0x7fd1d4103a70 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.660+0000 7fd1d1ffb700 1 -- 172.21.15.82:0/1344311338 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd1c8026020 con 0x7fd1d4103a70 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd1c4038760 msgr2=0x7fd1c403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:10.071 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd1c4038760 0x7fd1c403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0006fd0 tx=0x7fd1c0006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 msgr2=0x7fd1d418c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d418c7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800a240 tx=0x7fd1c80041d0 comp rx=0 tx=0).stop 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 shutdown_connections 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd1c4038760 0x7fd1c403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 --2- 172.21.15.82:0/1344311338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d4103a70 0x7fd1d418c7b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 >> 172.21.15.82:0/1344311338 conn(0x7fd1d40ff120 msgr2=0x7fd1d418d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:10.072 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 shutdown_connections 2023-07-14T19:21:10.073 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:09.662+0000 7fd1dae7a700 1 -- 172.21.15.82:0/1344311338 wait complete. 2023-07-14T19:21:10.074 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-14T19:21:10.074 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout disabled 2023-07-14T19:21:10.081 INFO:teuthology.orchestra.run.smithi082.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-14T19:21:10.081 INFO:teuthology.orchestra.run.smithi082.stdout:systemctl: stdout inactive 2023-07-14T19:21:10.081 INFO:teuthology.orchestra.run.smithi082.stdout:firewalld.service is not enabled 2023-07-14T19:21:10.081 INFO:teuthology.orchestra.run.smithi082.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-14T19:21:10.082 INFO:teuthology.orchestra.run.smithi082.stdout:Ceph Dashboard is now available at: 2023-07-14T19:21:10.083 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:10.083 INFO:teuthology.orchestra.run.smithi082.stdout: URL: https://smithi082:8443/ 2023-07-14T19:21:10.083 INFO:teuthology.orchestra.run.smithi082.stdout: User: admin 2023-07-14T19:21:10.083 INFO:teuthology.orchestra.run.smithi082.stdout: Password: t4sdnpnkwe 2023-07-14T19:21:10.083 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:10.084 INFO:teuthology.orchestra.run.smithi082.stdout:Saving cluster configuration to /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config directory 2023-07-14T19:21:10.084 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling autotune for osd_memory_target 2023-07-14T19:21:10.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:10 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1344311338' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-14T19:21:11.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa8655ac700 1 Processor -- start 2023-07-14T19:21:11.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa8655ac700 1 -- start start 2023-07-14T19:21:11.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa8655ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa860108320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:11.131 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa8655ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860079d20 con 0x7fa860105f40 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa85effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa860108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa85effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa860108320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39736/0 (socket says 172.21.15.82:39736) 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.601+0000 7fa85effd700 1 -- 172.21.15.82:0/3227486564 learned_addr learned my addr 172.21.15.82:0/3227486564 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa85effd700 1 -- 172.21.15.82:0/3227486564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860079e60 con 0x7fa860105f40 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa85effd700 1 --2- 172.21.15.82:0/3227486564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa860108320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098a0 tx=0x7fa854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcf46cc1b2787fb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa85dffb700 1 -- 172.21.15.82:0/3227486564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa854004160 con 0x7fa860105f40 2023-07-14T19:21:11.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa85dffb700 1 -- 172.21.15.82:0/3227486564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8540042c0 con 0x7fa860105f40 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa85dffb700 1 -- 172.21.15.82:0/3227486564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8540119b0 con 0x7fa860105f40 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 -- 172.21.15.82:0/3227486564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 msgr2=0x7fa860108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 --2- 172.21.15.82:0/3227486564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa860108320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa8540098a0 tx=0x7fa854009bb0 comp rx=0 tx=0).stop 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 -- 172.21.15.82:0/3227486564 shutdown_connections 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 --2- 172.21.15.82:0/3227486564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa860108320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 -- 172.21.15.82:0/3227486564 >> 172.21.15.82:0/3227486564 conn(0x7fa8600ff9a0 msgr2=0x7fa860101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 -- 172.21.15.82:0/3227486564 shutdown_connections 2023-07-14T19:21:11.133 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.602+0000 7fa8655ac700 1 -- 172.21.15.82:0/3227486564 wait complete. 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa8655ac700 1 Processor -- start 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa8655ac700 1 -- start start 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa8655ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa8601990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa8655ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860199610 con 0x7fa860105f40 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa85effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa8601990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa85effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa8601990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39738/0 (socket says 172.21.15.82:39738) 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.603+0000 7fa85effd700 1 -- 172.21.15.82:0/105096660 learned_addr learned my addr 172.21.15.82:0/105096660 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:11.134 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa85effd700 1 -- 172.21.15.82:0/105096660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa854009550 con 0x7fa860105f40 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa85effd700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa8601990d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa854003e60 tx=0x7fa854011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa854003970 con 0x7fa860105f40 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa854003ad0 con 0x7fa860105f40 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa854019590 con 0x7fa860105f40 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860199870 con 0x7fa860105f40 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.604+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa860199cf0 con 0x7fa860105f40 2023-07-14T19:21:11.135 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.605+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa854019800 con 0x7fa860105f40 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.605+0000 7fa847fff700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa848038740 0x7fa84803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.605+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa85404c920 con 0x7fa860105f40 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.605+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8601094b0 con 0x7fa860105f40 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.605+0000 7fa85e7fc700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa848038740 0x7fa84803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.605+0000 7fa85e7fc700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa848038740 0x7fa84803abf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa850006fd0 tx=0x7fa850006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.610+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa854003c40 con 0x7fa860105f40 2023-07-14T19:21:11.136 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.797+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa86019ca90 con 0x7fa860105f40 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.798+0000 7fa847fff700 1 -- 172.21.15.82:0/105096660 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa854023b40 con 0x7fa860105f40 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa848038740 msgr2=0x7fa84803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa848038740 0x7fa84803abf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa850006fd0 tx=0x7fa850006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 msgr2=0x7fa8601990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa8601990d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa854003e60 tx=0x7fa854011aa0 comp rx=0 tx=0).stop 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 shutdown_connections 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa848038740 0x7fa84803abf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:11.137 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 --2- 172.21.15.82:0/105096660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa860105f40 0x7fa8601990d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:11.138 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 >> 172.21.15.82:0/105096660 conn(0x7fa8600ff9a0 msgr2=0x7fa860100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:11.138 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 shutdown_connections 2023-07-14T19:21:11.138 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:10.800+0000 7fa8655ac700 1 -- 172.21.15.82:0/105096660 wait complete. 2023-07-14T19:21:12.273 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f686d417700 1 Processor -- start 2023-07-14T19:21:12.273 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f686d417700 1 -- start start 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f686d417700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f6868104100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f686d417700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868079d60 con 0x7f6868103cf0 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f6866ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f6868104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f6866ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f6868104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39742/0 (socket says 172.21.15.82:39742) 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.574+0000 7f6866ffd700 1 -- 172.21.15.82:0/452679314 learned_addr learned my addr 172.21.15.82:0/452679314 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f6866ffd700 1 -- 172.21.15.82:0/452679314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868079ea0 con 0x7f6868103cf0 2023-07-14T19:21:12.274 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f6866ffd700 1 --2- 172.21.15.82:0/452679314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f6868104100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f685c0098a0 tx=0x7f685c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e592fb34130bf6a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f6865ffb700 1 -- 172.21.15.82:0/452679314 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c004160 con 0x7f6868103cf0 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f6865ffb700 1 -- 172.21.15.82:0/452679314 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f685c0042c0 con 0x7f6868103cf0 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f6865ffb700 1 -- 172.21.15.82:0/452679314 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c0119b0 con 0x7f6868103cf0 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f686d417700 1 -- 172.21.15.82:0/452679314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 msgr2=0x7f6868104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f686d417700 1 --2- 172.21.15.82:0/452679314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f6868104100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f685c0098a0 tx=0x7f685c009bb0 comp rx=0 tx=0).stop 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f686d417700 1 -- 172.21.15.82:0/452679314 shutdown_connections 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f686d417700 1 --2- 172.21.15.82:0/452679314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f6868104100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:12.275 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.575+0000 7f686d417700 1 -- 172.21.15.82:0/452679314 >> 172.21.15.82:0/452679314 conn(0x7f68680ff9a0 msgr2=0x7f6868101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.576+0000 7f686d417700 1 -- 172.21.15.82:0/452679314 shutdown_connections 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.576+0000 7f686d417700 1 -- 172.21.15.82:0/452679314 wait complete. 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.576+0000 7f686d417700 1 Processor -- start 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f686d417700 1 -- start start 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f686d417700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f686819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f686d417700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686819b870 con 0x7f6868103cf0 2023-07-14T19:21:12.276 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f6866ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f686819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f6866ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f686819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39744/0 (socket says 172.21.15.82:39744) 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f6866ffd700 1 -- 172.21.15.82:0/274813505 learned_addr learned my addr 172.21.15.82:0/274813505 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f6866ffd700 1 -- 172.21.15.82:0/274813505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c009550 con 0x7f6868103cf0 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f6866ffd700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f686819b330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f685c000c00 tx=0x7f685c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c0039f0 con 0x7f6868103cf0 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f685c003b50 con 0x7f6868103cf0 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686819ba70 con 0x7f6868103cf0 2023-07-14T19:21:12.277 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c01a780 con 0x7f6868103cf0 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.577+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686819be90 con 0x7f6868103cf0 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.578+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f685c01a9f0 con 0x7f6868103cf0 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.579+0000 7f684ffff700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6850038850 0x7f685003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.579+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f685c04cb30 con 0x7f6868103cf0 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.579+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6868194b20 con 0x7f6868103cf0 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.579+0000 7f68667fc700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6850038850 0x7f685003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:12.278 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.579+0000 7f68667fc700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6850038850 0x7f685003ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6858006fd0 tx=0x7f6858006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.583+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f685c01acf0 con 0x7f6868103cf0 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.872+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f686802cb90 con 0x7f6868103cf0 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.877+0000 7f684ffff700 1 -- 172.21.15.82:0/274813505 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f685c02c350 con 0x7f6868103cf0 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.879+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6850038850 msgr2=0x7f685003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.879+0000 7f686d417700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6850038850 0x7f685003ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6858006fd0 tx=0x7f6858006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.879+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 msgr2=0x7f686819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.879+0000 7f686d417700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f686819b330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f685c000c00 tx=0x7f685c011a30 comp rx=0 tx=0).stop 2023-07-14T19:21:12.279 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.880+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 shutdown_connections 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.880+0000 7f686d417700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6850038850 0x7f685003ad00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.880+0000 7f686d417700 1 --2- 172.21.15.82:0/274813505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6868103cf0 0x7f686819b330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.880+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 >> 172.21.15.82:0/274813505 conn(0x7f68680ff9a0 msgr2=0x7f686818f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.880+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 shutdown_connections 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr 2023-07-14T19:21:11.880+0000 7f686d417700 1 -- 172.21.15.82:0/274813505 wait complete. 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-14T19:21:12.280 INFO:teuthology.orchestra.run.smithi082.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout:Or, if you are only running a single cluster on this host: 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.281 INFO:teuthology.orchestra.run.smithi082.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout: ceph telemetry on 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout:For more information see: 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:12.282 INFO:teuthology.orchestra.run.smithi082.stdout:Bootstrap complete. 2023-07-14T19:21:12.305 INFO:tasks.cephadm:Fetching config... 2023-07-14T19:21:12.306 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:12.306 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-14T19:21:12.364 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-14T19:21:12.365 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:12.365 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-14T19:21:12.421 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-14T19:21:12.422 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:12.422 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-14T19:21:12.489 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:12 smithi082 ceph-mon[127461]: mgrmap e13: x(active, since 6s) 2023-07-14T19:21:12.490 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:12 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/274813505' entity='client.admin' 2023-07-14T19:21:12.491 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-14T19:21:12.492 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:12.492 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-14T19:21:12.549 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-14T19:21:12.550 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T19:21:12.629 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:12.642 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T19:21:12.688 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:12.700 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-14T19:21:12.747 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJcJP87D3brBi/LQxC+UyeTahLaEEYyST5hvC1zNx3wFP38amk4m5xQB7F3C4v/uS9A7jjnqkhcC44m+HFogfTjjXF1BjvaM9DqYWIho14uWgFOx1+h3hco09sAc4Lt2CeKUzYVkd64C1RYhrR8Bc9iFxdNM7wRM0zK2VJZm3LT4AhUtgZdJEs1HyFxltNM5TK9FxPzvIonekQwbts/9VnIkKMEBGvy1nsbSSDndMnASlY5fwv9QyACnIXLhSobNqbvpFcuUB/FAbZh83G7DmFmysmOsIHsM7aWUZtmGw5DLxyRk372aCobjp5kRcQLmKzFp2zQ7rFZfd0hQaU4nuWPir/013ClQACX0Tkz1RWSXkIZlbHAIhWkd3q5w8/AK46h2a6H3Afn3gyo0oLaVAffHL2sT3i6rU9QXOpq5lA7arsS86JM1i4QZbSbMU+Ce8UF2ro+FbHj4+V6OQPw4JuA9UzcmHhTS4fp5UBHVk0cgFmbPazCmKU5WouJLlb9w8= ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:12.759 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-14T19:21:13.525 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:14.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.893+0000 7f8d6adac700 1 -- 172.21.15.82:0/1574050698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 msgr2=0x7f8d640fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:14.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.893+0000 7f8d6adac700 1 --2- 172.21.15.82:0/1574050698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d640fea20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009910 tx=0x7f8d58009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:14.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 -- 172.21.15.82:0/1574050698 shutdown_connections 2023-07-14T19:21:14.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 --2- 172.21.15.82:0/1574050698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d640fea20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 -- 172.21.15.82:0/1574050698 >> 172.21.15.82:0/1574050698 conn(0x7f8d640f9ba0 msgr2=0x7f8d640fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 -- 172.21.15.82:0/1574050698 shutdown_connections 2023-07-14T19:21:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 -- 172.21.15.82:0/1574050698 wait complete. 2023-07-14T19:21:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 Processor -- start 2023-07-14T19:21:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.894+0000 7f8d6adac700 1 -- start start 2023-07-14T19:21:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d6adac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d64193760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d6adac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d64193ca0 con 0x7f8d640fe610 2023-07-14T19:21:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d68b48700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d64193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d68b48700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d64193760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39750/0 (socket says 172.21.15.82:39750) 2023-07-14T19:21:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d68b48700 1 -- 172.21.15.82:0/550444070 learned_addr learned my addr 172.21.15.82:0/550444070 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d68b48700 1 -- 172.21.15.82:0/550444070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580095f0 con 0x7f8d640fe610 2023-07-14T19:21:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d68b48700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d64193760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d58006010 tx=0x7f8d580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d5801d070 con 0x7f8d640fe610 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d580044e0 con 0x7f8d640fe610 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d64193ea0 con 0x7f8d640fe610 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.895+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d580039b0 con 0x7f8d640fe610 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.896+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d641942e0 con 0x7f8d640fe610 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.896+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8d58003c70 con 0x7f8d640fe610 2023-07-14T19:21:14.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.896+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6418d1e0 con 0x7f8d640fe610 2023-07-14T19:21:14.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.896+0000 7f8d61ffb700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d540387b0 0x7f8d5403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:14.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.896+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d5804ca70 con 0x7f8d640fe610 2023-07-14T19:21:14.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.897+0000 7f8d63fff700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d540387b0 0x7f8d5403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:14.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.897+0000 7f8d63fff700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d540387b0 0x7f8d5403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d50006fd0 tx=0x7f8d50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:14.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:14.901+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8d5802a950 con 0x7f8d640fe610 2023-07-14T19:21:15.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.090+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8d6402c8e0 con 0x7f8d640fe610 2023-07-14T19:21:15.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.093+0000 7f8d61ffb700 1 -- 172.21.15.82:0/550444070 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8d5802a360 con 0x7f8d640fe610 2023-07-14T19:21:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d540387b0 msgr2=0x7f8d5403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d540387b0 0x7f8d5403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d50006fd0 tx=0x7f8d50006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:15.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 msgr2=0x7f8d64193760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:15.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d64193760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d58006010 tx=0x7f8d580057c0 comp rx=0 tx=0).stop 2023-07-14T19:21:15.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 shutdown_connections 2023-07-14T19:21:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d540387b0 0x7f8d5403ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 --2- 172.21.15.82:0/550444070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d640fe610 0x7f8d64193760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 >> 172.21.15.82:0/550444070 conn(0x7f8d640f9ba0 msgr2=0x7f8d640fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 shutdown_connections 2023-07-14T19:21:15.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:15.095+0000 7f8d6adac700 1 -- 172.21.15.82:0/550444070 wait complete. 2023-07-14T19:21:15.672 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-14T19:21:15.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-14T19:21:16.224 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:16.402 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/550444070' entity='client.admin' 2023-07-14T19:21:16.403 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:16.403 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:16.403 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:21:16.403 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:16.403 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:16.404 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:17.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/94001095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 msgr2=0x7f4a580fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:17.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 --2- 172.21.15.82:0/94001095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a580fea20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009910 tx=0x7f4a48009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:17.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/94001095 shutdown_connections 2023-07-14T19:21:17.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 --2- 172.21.15.82:0/94001095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a580fea20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:17.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/94001095 >> 172.21.15.82:0/94001095 conn(0x7f4a580f9ba0 msgr2=0x7f4a580fbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:17.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/94001095 shutdown_connections 2023-07-14T19:21:17.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.430+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/94001095 wait complete. 2023-07-14T19:21:17.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5f8fd700 1 Processor -- start 2023-07-14T19:21:17.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5f8fd700 1 -- start start 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5f8fd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a58193780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5f8fd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a58193cc0 con 0x7f4a580fe610 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5d699700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a58193780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5d699700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a58193780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39756/0 (socket says 172.21.15.82:39756) 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5d699700 1 -- 172.21.15.82:0/4023768088 learned_addr learned my addr 172.21.15.82:0/4023768088 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5d699700 1 -- 172.21.15.82:0/4023768088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480095f0 con 0x7f4a580fe610 2023-07-14T19:21:17.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.431+0000 7f4a5d699700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a58193780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a480057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:17.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.432+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a4801d070 con 0x7f4a580fe610 2023-07-14T19:21:17.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.432+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a480044e0 con 0x7f4a580fe610 2023-07-14T19:21:17.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.432+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a58193ec0 con 0x7f4a580fe610 2023-07-14T19:21:17.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.432+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a480039b0 con 0x7f4a580fe610 2023-07-14T19:21:17.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.432+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a58194300 con 0x7f4a580fe610 2023-07-14T19:21:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.432+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4a48003c20 con 0x7f4a580fe610 2023-07-14T19:21:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.433+0000 7f4a4e7fc700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4a44038800 0x7f4a4403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.433+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a4804c770 con 0x7f4a580fe610 2023-07-14T19:21:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.433+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5818d1e0 con 0x7f4a580fe610 2023-07-14T19:21:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.433+0000 7f4a5ce98700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4a44038800 0x7f4a4403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:17.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.433+0000 7f4a5ce98700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4a44038800 0x7f4a4403acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4a54006fd0 tx=0x7f4a54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:17.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.437+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4a4802a930 con 0x7f4a580fe610 2023-07-14T19:21:17.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.626+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a58060b10 con 0x7f4a44038800 2023-07-14T19:21:17.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.629+0000 7f4a4e7fc700 1 -- 172.21.15.82:0/4023768088 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4a58060b10 con 0x7f4a44038800 2023-07-14T19:21:17.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4a44038800 msgr2=0x7f4a4403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:17.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4a44038800 0x7f4a4403acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4a54006fd0 tx=0x7f4a54006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:17.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 msgr2=0x7f4a58193780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a58193780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006010 tx=0x7f4a480057c0 comp rx=0 tx=0).stop 2023-07-14T19:21:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 shutdown_connections 2023-07-14T19:21:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4a44038800 0x7f4a4403acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 --2- 172.21.15.82:0/4023768088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a580fe610 0x7f4a58193780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 >> 172.21.15.82:0/4023768088 conn(0x7f4a580f9ba0 msgr2=0x7f4a580fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:17.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 shutdown_connections 2023-07-14T19:21:17.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:17.631+0000 7f4a5f8fd700 1 -- 172.21.15.82:0/4023768088 wait complete. 2023-07-14T19:21:18.072 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2023-07-14T19:21:18.073 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:21:18.073 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2023-07-14T19:21:18.095 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:21:18.095 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:18.154 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2023-07-14T19:21:18.154 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch host add smithi133 2023-07-14T19:21:18.718 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:18.902 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:18.902 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:18.902 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:18.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:19.756 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:19 smithi082 ceph-mon[127461]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:19.756 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:19 smithi082 ceph-mon[127461]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:21:19.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2724082199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 msgr2=0x7fbcc8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 --2- 172.21.15.82:0/2724082199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc8104a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009910 tx=0x7fbcb8009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2724082199 shutdown_connections 2023-07-14T19:21:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 --2- 172.21.15.82:0/2724082199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc8104a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:19.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2724082199 >> 172.21.15.82:0/2724082199 conn(0x7fbcc80fa140 msgr2=0x7fbcc80fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:19.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2724082199 shutdown_connections 2023-07-14T19:21:19.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.897+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2724082199 wait complete. 2023-07-14T19:21:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbccd1f0700 1 Processor -- start 2023-07-14T19:21:19.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbccd1f0700 1 -- start start 2023-07-14T19:21:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbccd1f0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbccd1f0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc819ada0 con 0x7fbcc8104660 2023-07-14T19:21:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbcc6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39762/0 (socket says 172.21.15.82:39762) 2023-07-14T19:21:19.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.898+0000 7fbcc6d9d700 1 -- 172.21.15.82:0/2421325805 learned_addr learned my addr 172.21.15.82:0/2421325805 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbcc6d9d700 1 -- 172.21.15.82:0/2421325805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80095f0 con 0x7fbcc8104660 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbcc6d9d700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc819a860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcb801d070 con 0x7fbcc8104660 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb800bc10 con 0x7fbcc8104660 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc819b000 con 0x7fbcc8104660 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcb8003910 con 0x7fbcc8104660 2023-07-14T19:21:19.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.899+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc81948e0 con 0x7fbcc8104660 2023-07-14T19:21:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.900+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbcb8003bd0 con 0x7fbcc8104660 2023-07-14T19:21:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.900+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc8061c70 con 0x7fbcc8104660 2023-07-14T19:21:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.900+0000 7fbcaffff700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbcb00387b0 0x7fbcb003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.900+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbcb804c4f0 con 0x7fbcc8104660 2023-07-14T19:21:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.900+0000 7fbcc659c700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbcb00387b0 0x7fbcb003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:19.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.900+0000 7fbcc659c700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbcb00387b0 0x7fbcb003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc006fd0 tx=0x7fbcbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:19.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:19.905+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fbcb8029950 con 0x7fbcc8104660 2023-07-14T19:21:20.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:20.094+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcc80fb750 con 0x7fbcb00387b0 2023-07-14T19:21:21.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:21 smithi082 ceph-mon[127461]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:21.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:21 smithi082 ceph-mon[127461]: Deploying cephadm binary to smithi133 2023-07-14T19:21:22.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.358+0000 7fbcaffff700 1 -- 172.21.15.82:0/2421325805 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbcc80fb750 con 0x7fbcb00387b0 2023-07-14T19:21:22.361 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi133' with addr '172.21.15.133' 2023-07-14T19:21:22.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbcb00387b0 msgr2=0x7fbcb003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:22.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbcb00387b0 0x7fbcb003ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc006fd0 tx=0x7fbcbc006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:22.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 msgr2=0x7fbcc819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:22.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc819a860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8006010 tx=0x7fbcb8004ff0 comp rx=0 tx=0).stop 2023-07-14T19:21:22.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 shutdown_connections 2023-07-14T19:21:22.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbcb00387b0 0x7fbcb003ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:22.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 --2- 172.21.15.82:0/2421325805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcc8104660 0x7fbcc819a860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:22.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 >> 172.21.15.82:0/2421325805 conn(0x7fbcc80fa140 msgr2=0x7fbcc80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:22.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 shutdown_connections 2023-07-14T19:21:22.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:22.361+0000 7fbccd1f0700 1 -- 172.21.15.82:0/2421325805 wait complete. 2023-07-14T19:21:22.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T19:21:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:23.706 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:23 smithi082 ceph-mon[127461]: Added host smithi133 2023-07-14T19:21:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:23.707 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:24.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 -- 172.21.15.82:0/3018155848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 msgr2=0x7f19e4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:24.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 --2- 172.21.15.82:0/3018155848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e4104a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009910 tx=0x7f19d4009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:24.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 -- 172.21.15.82:0/3018155848 shutdown_connections 2023-07-14T19:21:24.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 --2- 172.21.15.82:0/3018155848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e4104a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 -- 172.21.15.82:0/3018155848 >> 172.21.15.82:0/3018155848 conn(0x7f19e40fa140 msgr2=0x7f19e40fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 -- 172.21.15.82:0/3018155848 shutdown_connections 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.507+0000 7f19ec6ad700 1 -- 172.21.15.82:0/3018155848 wait complete. 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ec6ad700 1 Processor -- start 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ec6ad700 1 -- start start 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ec6ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:24.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ec6ad700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419adb0 con 0x7f19e4104660 2023-07-14T19:21:24.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ea449700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:24.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ea449700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39788/0 (socket says 172.21.15.82:39788) 2023-07-14T19:21:24.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ea449700 1 -- 172.21.15.82:0/921768280 learned_addr learned my addr 172.21.15.82:0/921768280 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:24.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.508+0000 7f19ea449700 1 -- 172.21.15.82:0/921768280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40095f0 con 0x7f19e4104660 2023-07-14T19:21:24.513 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.509+0000 7f19ea449700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e419a870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f19d4004f40 tx=0x7f19d4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:24.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.509+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19d401d070 con 0x7f19e4104660 2023-07-14T19:21:24.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.509+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d4004370 con 0x7f19e4104660 2023-07-14T19:21:24.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.509+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e419b010 con 0x7f19e4104660 2023-07-14T19:21:24.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.509+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19d40039b0 con 0x7f19e4104660 2023-07-14T19:21:24.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.509+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e4194990 con 0x7f19e4104660 2023-07-14T19:21:24.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.510+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f19d4003c70 con 0x7f19e4104660 2023-07-14T19:21:24.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.510+0000 7f19db7fe700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d00387b0 0x7f19d003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:24.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.510+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19d404c570 con 0x7f19e4104660 2023-07-14T19:21:24.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.510+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4061c70 con 0x7f19e4104660 2023-07-14T19:21:24.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.510+0000 7f19e9c48700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d00387b0 0x7f19d003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:24.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.510+0000 7f19e9c48700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d00387b0 0x7f19d003ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:24.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.515+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f19d4026020 con 0x7f19e4104660 2023-07-14T19:21:24.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.706+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19e4103880 con 0x7f19d00387b0 2023-07-14T19:21:24.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.707+0000 7f19db7fe700 1 -- 172.21.15.82:0/921768280 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f19e4103880 con 0x7f19d00387b0 2023-07-14T19:21:24.709 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:24.709 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2023-07-14T19:21:24.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d00387b0 msgr2=0x7f19d003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:24.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d00387b0 0x7f19d003ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:24.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 msgr2=0x7f19e419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:24.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e419a870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f19d4004f40 tx=0x7f19d4005020 comp rx=0 tx=0).stop 2023-07-14T19:21:24.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 shutdown_connections 2023-07-14T19:21:24.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d00387b0 0x7f19d003ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:24.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 --2- 172.21.15.82:0/921768280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19e4104660 0x7f19e419a870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:24.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 >> 172.21.15.82:0/921768280 conn(0x7f19e40fa140 msgr2=0x7f19e4102160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:24.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 shutdown_connections 2023-07-14T19:21:24.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:24.709+0000 7f19ec6ad700 1 -- 172.21.15.82:0/921768280 wait complete. 2023-07-14T19:21:24.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:24 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:25.073 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2023-07-14T19:21:25.073 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:21:25.074 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2023-07-14T19:21:25.095 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:21:25.096 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:25.155 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2023-07-14T19:21:25.155 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch host add smithi161 2023-07-14T19:21:25.704 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:26.065 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:25 smithi082 ceph-mon[127461]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:26.066 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:25 smithi082 ceph-mon[127461]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:21:26.066 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:26.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 -- 172.21.15.82:0/110064572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 msgr2=0x7fc44c105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:26.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 --2- 172.21.15.82:0/110064572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c105af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc440009910 tx=0x7fc440009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 -- 172.21.15.82:0/110064572 shutdown_connections 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 --2- 172.21.15.82:0/110064572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c105af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 -- 172.21.15.82:0/110064572 >> 172.21.15.82:0/110064572 conn(0x7fc44c0fb220 msgr2=0x7fc44c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 -- 172.21.15.82:0/110064572 shutdown_connections 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 -- 172.21.15.82:0/110064572 wait complete. 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.854+0000 7fc451a10700 1 Processor -- start 2023-07-14T19:21:26.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc451a10700 1 -- start start 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc451a10700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc451a10700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c074910 con 0x7fc44c105720 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc44affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc44affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39794/0 (socket says 172.21.15.82:39794) 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc44affd700 1 -- 172.21.15.82:0/1079435812 learned_addr learned my addr 172.21.15.82:0/1079435812 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc44affd700 1 -- 172.21.15.82:0/1079435812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400095f0 con 0x7fc44c105720 2023-07-14T19:21:26.859 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc44affd700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c072940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc440004f40 tx=0x7fc440005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:26.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.855+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc44001d070 con 0x7fc44c105720 2023-07-14T19:21:26.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.856+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc44000be20 con 0x7fc44c105720 2023-07-14T19:21:26.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.856+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc440022d00 con 0x7fc44c105720 2023-07-14T19:21:26.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.856+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c072ee0 con 0x7fc44c105720 2023-07-14T19:21:26.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.856+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c073360 con 0x7fc44c105720 2023-07-14T19:21:26.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.856+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc44000b960 con 0x7fc44c105720 2023-07-14T19:21:26.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.857+0000 7fc450a0e700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4340387b0 0x7fc43403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:26.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.857+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c104a10 con 0x7fc44c105720 2023-07-14T19:21:26.861 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.857+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc44004c9e0 con 0x7fc44c105720 2023-07-14T19:21:26.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.857+0000 7fc44a7fc700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4340387b0 0x7fc43403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:26.862 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.857+0000 7fc44a7fc700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4340387b0 0x7fc43403ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006fd0 tx=0x7fc43c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:26.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:26.861+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc44002a540 con 0x7fc44c105720 2023-07-14T19:21:27.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:27.053+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) v1 -- 0x7fc44c0735c0 con 0x7fc4340387b0 2023-07-14T19:21:28.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:21:28.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:28.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:29.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.373+0000 7fc450a0e700 1 -- 172.21.15.82:0/1079435812 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fc44c0735c0 con 0x7fc4340387b0 2023-07-14T19:21:29.375 INFO:teuthology.orchestra.run.smithi082.stdout:Added host 'smithi161' with addr '172.21.15.161' 2023-07-14T19:21:29.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4340387b0 msgr2=0x7fc43403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:29.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4340387b0 0x7fc43403ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006fd0 tx=0x7fc43c006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:29.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 msgr2=0x7fc44c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c072940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc440004f40 tx=0x7fc440005020 comp rx=0 tx=0).stop 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 shutdown_connections 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4340387b0 0x7fc43403ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 --2- 172.21.15.82:0/1079435812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44c105720 0x7fc44c072940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 >> 172.21.15.82:0/1079435812 conn(0x7fc44c0fb220 msgr2=0x7fc44c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 shutdown_connections 2023-07-14T19:21:29.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:29.375+0000 7fc451a10700 1 -- 172.21.15.82:0/1079435812 wait complete. 2023-07-14T19:21:29.399 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:21:29.399 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:29.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: Deploying cephadm binary to smithi161 2023-07-14T19:21:29.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:29.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:21:29.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:29.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:29.400 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:29.764 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch host ls --format=json 2023-07-14T19:21:30.312 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:30.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:30 smithi082 ceph-mon[127461]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:30.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:30.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:30.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:31.156 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:31 smithi082 ceph-mon[127461]: Added host smithi161 2023-07-14T19:21:31.157 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:31.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 -- 172.21.15.82:0/3323982457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 msgr2=0x7f7c18104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 --2- 172.21.15.82:0/3323982457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c18104a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009910 tx=0x7f7c00009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 -- 172.21.15.82:0/3323982457 shutdown_connections 2023-07-14T19:21:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 --2- 172.21.15.82:0/3323982457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c18104a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:31.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 -- 172.21.15.82:0/3323982457 >> 172.21.15.82:0/3323982457 conn(0x7f7c180fa140 msgr2=0x7f7c180fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 -- 172.21.15.82:0/3323982457 shutdown_connections 2023-07-14T19:21:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 -- 172.21.15.82:0/3323982457 wait complete. 2023-07-14T19:21:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.585+0000 7f7c1e850700 1 Processor -- start 2023-07-14T19:21:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c1e850700 1 -- start start 2023-07-14T19:21:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c1e850700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c181958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:31.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c1e850700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c18195e20 con 0x7f7c18104660 2023-07-14T19:21:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c17fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c181958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c17fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c181958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39820/0 (socket says 172.21.15.82:39820) 2023-07-14T19:21:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c17fff700 1 -- 172.21.15.82:0/2416578844 learned_addr learned my addr 172.21.15.82:0/2416578844 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:31.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c17fff700 1 -- 172.21.15.82:0/2416578844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000095f0 con 0x7f7c18104660 2023-07-14T19:21:31.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c17fff700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c181958e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7c00004f40 tx=0x7f7c00005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:31.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c0001d070 con 0x7f7c18104660 2023-07-14T19:21:31.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c00004370 con 0x7f7c18104660 2023-07-14T19:21:31.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.586+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c000039b0 con 0x7f7c18104660 2023-07-14T19:21:31.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.587+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c18196080 con 0x7f7c18104660 2023-07-14T19:21:31.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.587+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c18196580 con 0x7f7c18104660 2023-07-14T19:21:31.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.587+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f7c00003c70 con 0x7f7c18104660 2023-07-14T19:21:31.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.587+0000 7f7c157fa700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c040387b0 0x7f7c0403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:31.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.587+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c0004c550 con 0x7f7c18104660 2023-07-14T19:21:31.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.588+0000 7f7c177fe700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c040387b0 0x7f7c0403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:31.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.588+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18103900 con 0x7f7c18104660 2023-07-14T19:21:31.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.588+0000 7f7c177fe700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c040387b0 0x7f7c0403ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:31.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.593+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7c00026020 con 0x7f7c18104660 2023-07-14T19:21:31.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.792+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c18068750 con 0x7f7c040387b0 2023-07-14T19:21:31.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.793+0000 7f7c157fa700 1 -- 172.21.15.82:0/2416578844 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f7c18068750 con 0x7f7c040387b0 2023-07-14T19:21:31.795 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:21:31.795 INFO:teuthology.orchestra.run.smithi082.stdout:[{"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2023-07-14T19:21:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c040387b0 msgr2=0x7f7c0403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c040387b0 0x7f7c0403ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006fd0 tx=0x7f7c08006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 msgr2=0x7f7c181958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c181958e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7c00004f40 tx=0x7f7c00005020 comp rx=0 tx=0).stop 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 shutdown_connections 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c040387b0 0x7f7c0403ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 --2- 172.21.15.82:0/2416578844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c18104660 0x7f7c181958e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 >> 172.21.15.82:0/2416578844 conn(0x7f7c180fa140 msgr2=0x7f7c180fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:31.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 shutdown_connections 2023-07-14T19:21:31.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:31.796+0000 7f7c1e850700 1 -- 172.21.15.82:0/2416578844 wait complete. 2023-07-14T19:21:32.155 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:32 smithi082 ceph-mon[127461]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:32.156 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:32 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:32.157 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-14T19:21:32.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd crush tunables default 2023-07-14T19:21:32.707 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:33 smithi082 ceph-mon[127461]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:21:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/1334307772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 msgr2=0x7fcb28104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 --2- 172.21.15.82:0/1334307772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28104a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c009910 tx=0x7fcb1c009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/1334307772 shutdown_connections 2023-07-14T19:21:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 --2- 172.21.15.82:0/1334307772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28104a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/1334307772 >> 172.21.15.82:0/1334307772 conn(0x7fcb280743e0 msgr2=0x7fcb280747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/1334307772 shutdown_connections 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.774+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/1334307772 wait complete. 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ecc5700 1 Processor -- start 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ecc5700 1 -- start start 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ecc5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ecc5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb28195f50 con 0x7fcb28104660 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ca61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ca61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39826/0 (socket says 172.21.15.82:39826) 2023-07-14T19:21:33.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.775+0000 7fcb2ca61700 1 -- 172.21.15.82:0/983947571 learned_addr learned my addr 172.21.15.82:0/983947571 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb2ca61700 1 -- 172.21.15.82:0/983947571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c0095f0 con 0x7fcb28104660 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb2ca61700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28195a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c004f40 tx=0x7fcb1c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb1c01d070 con 0x7fcb28104660 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb1c004370 con 0x7fcb28104660 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb281961b0 con 0x7fcb28104660 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb1c0039b0 con 0x7fcb28104660 2023-07-14T19:21:33.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.776+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb28196630 con 0x7fcb28104660 2023-07-14T19:21:33.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.777+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fcb1c003c70 con 0x7fcb28104660 2023-07-14T19:21:33.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.777+0000 7fcb25ffb700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb180387b0 0x7fcb1803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:33.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.777+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcb1c04c680 con 0x7fcb28104660 2023-07-14T19:21:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.777+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb28196880 con 0x7fcb28104660 2023-07-14T19:21:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.777+0000 7fcb27fff700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb180387b0 0x7fcb1803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.778+0000 7fcb27fff700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb180387b0 0x7fcb1803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb14006fd0 tx=0x7fcb14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.782+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fcb28196880 con 0x7fcb28104660 2023-07-14T19:21:33.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:33.971+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcb28103900 con 0x7fcb28104660 2023-07-14T19:21:34.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.035+0000 7fcb25ffb700 1 -- 172.21.15.82:0/983947571 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcb1c026070 con 0x7fcb28104660 2023-07-14T19:21:34.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb180387b0 msgr2=0x7fcb1803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:34.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb180387b0 0x7fcb1803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb14006fd0 tx=0x7fcb14006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:34.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 msgr2=0x7fcb28195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:34.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28195a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c004f40 tx=0x7fcb1c005020 comp rx=0 tx=0).stop 2023-07-14T19:21:34.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 shutdown_connections 2023-07-14T19:21:34.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb180387b0 0x7fcb1803ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:34.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 --2- 172.21.15.82:0/983947571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb28104660 0x7fcb28195a10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:34.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 >> 172.21.15.82:0/983947571 conn(0x7fcb280743e0 msgr2=0x7fcb28105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:34.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 shutdown_connections 2023-07-14T19:21:34.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:34.038+0000 7fcb2ecc5700 1 -- 172.21.15.82:0/983947571 wait complete. 2023-07-14T19:21:34.040 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2023-07-14T19:21:34.326 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:34 smithi082 ceph-mon[127461]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:34.327 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:34 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/983947571' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T19:21:34.350 INFO:tasks.cephadm:Adding mon.a on smithi082 2023-07-14T19:21:34.351 INFO:tasks.cephadm:Adding mon.b on smithi133 2023-07-14T19:21:34.352 INFO:tasks.cephadm:Adding mon.c on smithi161 2023-07-14T19:21:34.352 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch apply mon '3;smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c' 2023-07-14T19:21:34.955 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:35.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/983947571' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:21:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:35.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:36.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 -- 172.21.15.161:0/1374258928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 msgr2=0x7f7cd8105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:36.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 --2- 172.21.15.161:0/1374258928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8105b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009910 tx=0x7f7cc8009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:36.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 -- 172.21.15.161:0/1374258928 shutdown_connections 2023-07-14T19:21:36.291 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 --2- 172.21.15.161:0/1374258928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8105b10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 -- 172.21.15.161:0/1374258928 >> 172.21.15.161:0/1374258928 conn(0x7f7cd80fb260 msgr2=0x7f7cd80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 -- 172.21.15.161:0/1374258928 shutdown_connections 2023-07-14T19:21:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.288+0000 7f7cde658700 1 -- 172.21.15.161:0/1374258928 wait complete. 2023-07-14T19:21:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.289+0000 7f7cde658700 1 Processor -- start 2023-07-14T19:21:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.289+0000 7f7cde658700 1 -- start start 2023-07-14T19:21:36.292 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.289+0000 7f7cde658700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:36.293 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.289+0000 7f7cde658700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd8074910 con 0x7f7cd8105740 2023-07-14T19:21:36.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:36.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.161:45012/0 (socket says 172.21.15.161:45012) 2023-07-14T19:21:36.294 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd7fff700 1 -- 172.21.15.161:0/3874114208 learned_addr learned my addr 172.21.15.161:0/3874114208 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd7fff700 1 -- 172.21.15.161:0/3874114208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc80095f0 con 0x7f7cd8105740 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd7fff700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8072940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8004f40 tx=0x7f7cc8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc801d070 con 0x7f7cd8105740 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd8072ee0 con 0x7f7cd8105740 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cc800be20 con 0x7f7cd8105740 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc8022d00 con 0x7f7cd8105740 2023-07-14T19:21:36.295 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.290+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd8073360 con 0x7f7cd8105740 2023-07-14T19:21:36.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.291+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f7cc800b960 con 0x7f7cd8105740 2023-07-14T19:21:36.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.291+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd8104a30 con 0x7f7cd8105740 2023-07-14T19:21:36.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.292+0000 7f7cd57fa700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7cc00387b0 0x7f7cc003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:36.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.292+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7cc804c9e0 con 0x7f7cd8105740 2023-07-14T19:21:36.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.292+0000 7f7cd77fe700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7cc00387b0 0x7f7cc003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:36.296 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.292+0000 7f7cd77fe700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7cc00387b0 0x7f7cc003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:36.300 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.297+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7cc802a540 con 0x7f7cd8105740 2023-07-14T19:21:36.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:21:36.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:36.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:36.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:36.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:21:36.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:36.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:36.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:36 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:36.489 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.487+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cd80735c0 con 0x7f7cc00387b0 2023-07-14T19:21:36.498 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.494+0000 7f7cd57fa700 1 -- 172.21.15.161:0/3874114208 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7cd80735c0 con 0x7f7cc00387b0 2023-07-14T19:21:36.498 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2023-07-14T19:21:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7cc00387b0 msgr2=0x7f7cc003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7cc00387b0 0x7f7cc003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 msgr2=0x7f7cd8072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8072940 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8004f40 tx=0x7f7cc8005020 comp rx=0 tx=0).stop 2023-07-14T19:21:36.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 shutdown_connections 2023-07-14T19:21:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7cc00387b0 0x7f7cc003ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 --2- 172.21.15.161:0/3874114208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cd8105740 0x7f7cd8072940 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 >> 172.21.15.161:0/3874114208 conn(0x7f7cd80fb260 msgr2=0x7f7cd80fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 shutdown_connections 2023-07-14T19:21:36.500 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:36.496+0000 7f7cde658700 1 -- 172.21.15.161:0/3874114208 wait complete. 2023-07-14T19:21:36.925 DEBUG:teuthology.orchestra.run.smithi133:mon.b> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.b.service 2023-07-14T19:21:36.928 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.c.service 2023-07-14T19:21:36.932 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T19:21:36.932 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T19:21:36.961 INFO:journalctl@ceph.mon.c.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:21:36.964 INFO:journalctl@ceph.mon.b.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:21:37.815 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:37.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c;count:3 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: Deploying daemon mon.c on smithi161 2023-07-14T19:21:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:37 smithi082 ceph-mon[127461]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:39.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:39 smithi161 systemd[1]: Starting Ceph mon.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:21:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 -- 172.21.15.161:0/3741132836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 msgr2=0x7fa42c0fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:39.752 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 --2- 172.21.15.161:0/3741132836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c0fe950 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009910 tx=0x7fa41c009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 -- 172.21.15.161:0/3741132836 shutdown_connections 2023-07-14T19:21:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 --2- 172.21.15.161:0/3741132836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c0fe950 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:39.753 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 -- 172.21.15.161:0/3741132836 >> 172.21.15.161:0/3741132836 conn(0x7fa42c075040 msgr2=0x7fa42c075440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:39.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 -- 172.21.15.161:0/3741132836 shutdown_connections 2023-07-14T19:21:39.754 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.750+0000 7fa4324db700 1 -- 172.21.15.161:0/3741132836 wait complete. 2023-07-14T19:21:39.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.751+0000 7fa4324db700 1 Processor -- start 2023-07-14T19:21:39.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.751+0000 7fa4324db700 1 -- start start 2023-07-14T19:21:39.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.751+0000 7fa4324db700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:39.757 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.751+0000 7fa4324db700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c19ae40 con 0x7fa42c0fe580 2023-07-14T19:21:39.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.752+0000 7fa42bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:39.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.752+0000 7fa42bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c19a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.161:45018/0 (socket says 172.21.15.161:45018) 2023-07-14T19:21:39.758 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.752+0000 7fa42bfff700 1 -- 172.21.15.161:0/1306829400 learned_addr learned my addr 172.21.15.161:0/1306829400 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:21:39.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.752+0000 7fa42bfff700 1 -- 172.21.15.161:0/1306829400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0095f0 con 0x7fa42c0fe580 2023-07-14T19:21:39.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.752+0000 7fa42bfff700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c19a900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa41c004f40 tx=0x7fa41c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:39.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.752+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41c01d070 con 0x7fa42c0fe580 2023-07-14T19:21:39.759 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.753+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa41c004370 con 0x7fa42c0fe580 2023-07-14T19:21:39.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.753+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41c0039b0 con 0x7fa42c0fe580 2023-07-14T19:21:39.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.753+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c19b040 con 0x7fa42c0fe580 2023-07-14T19:21:39.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.753+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c194980 con 0x7fa42c0fe580 2023-07-14T19:21:39.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.754+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa41c003c20 con 0x7fa42c0fe580 2023-07-14T19:21:39.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.754+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c061c70 con 0x7fa42c0fe580 2023-07-14T19:21:39.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.754+0000 7fa4297fa700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa414038800 0x7fa41403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:39.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.754+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa41c04c710 con 0x7fa42c0fe580 2023-07-14T19:21:39.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.754+0000 7fa42b7fe700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa414038800 0x7fa41403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:39.761 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.755+0000 7fa42b7fe700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa414038800 0x7fa41403acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa420006fd0 tx=0x7fa420006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:39.762 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:39.759+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa41c029950 con 0x7fa42c0fe580 2023-07-14T19:21:39.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:39 smithi082 ceph-mon[127461]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.031+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa42c195340 con 0x7fa42c0fe580 2023-07-14T19:21:40.033 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.032+0000 7fa4297fa700 1 -- 172.21.15.161:0/1306829400 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fa41c026030 con 0x7fa42c0fe580 2023-07-14T19:21:40.034 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:21:40.034 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","modified":"2023-07-14T19:20:10.680493Z","created":"2023-07-14T19:20:10.680493Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-14T19:21:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa414038800 msgr2=0x7fa41403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa414038800 0x7fa41403acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa420006fd0 tx=0x7fa420006e40 comp rx=0 tx=0).stop 2023-07-14T19:21:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 msgr2=0x7fa42c19a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c19a900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa41c004f40 tx=0x7fa41c005020 comp rx=0 tx=0).stop 2023-07-14T19:21:40.036 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 shutdown_connections 2023-07-14T19:21:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa414038800 0x7fa41403acb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 --2- 172.21.15.161:0/1306829400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0fe580 0x7fa42c19a900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 >> 172.21.15.161:0/1306829400 conn(0x7fa42c075040 msgr2=0x7fa42c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 shutdown_connections 2023-07-14T19:21:40.037 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:40.034+0000 7fa4324db700 1 -- 172.21.15.161:0/1306829400 wait complete. 2023-07-14T19:21:40.038 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2023-07-14T19:21:40.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:39 smithi161 podman[129117]: 2023-07-14T19:21:40.446 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T19:21:40.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev), process ceph-mon, pid 7 2023-07-14T19:21:40.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: pidfile_write: ignore empty --pid-file 2023-07-14T19:21:40.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: load: jerasure load: lrc 2023-07-14T19:21:40.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: RocksDB version: 7.9.2 2023-07-14T19:21:40.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Git sha 0 2023-07-14T19:21:40.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Compile date 2023-07-14 14:45:42 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: DB SUMMARY 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: DB Session ID: N4I4PEC5L51ASL23F6KO 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: CURRENT file: CURRENT 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-14T19:21:40.448 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.error_if_exists: 0 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.create_if_missing: 0 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.paranoid_checks: 1 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.env: 0x5586510cac60 2023-07-14T19:21:40.449 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.info_log: 0x55865359f2a0 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.statistics: (nil) 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.use_fsync: 0 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_log_file_size: 0 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T19:21:40.450 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.allow_fallocate: 1 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.use_direct_reads: 0 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T19:21:40.451 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T19:21:40.452 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.db_log_dir: 2023-07-14T19:21:40.452 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.wal_dir: 2023-07-14T19:21:40.452 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T19:21:40.453 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T19:21:40.453 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T19:21:40.453 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T19:21:40.454 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T19:21:40.454 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T19:21:40.454 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T19:21:40.454 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T19:21:40.454 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.write_buffer_manager: 0x5586534c03c0 2023-07-14T19:21:40.454 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T19:21:40.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T19:21:40.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T19:21:40.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T19:21:40.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T19:21:40.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T19:21:40.455 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.unordered_write: 0 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T19:21:40.456 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.row_cache: None 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.wal_filter: None 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.two_write_queues: 0 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.wal_compression: 0 2023-07-14T19:21:40.457 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.atomic_flush: 0 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.log_readahead_size: 0 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T19:21:40.458 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_background_jobs: 2 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_background_compactions: -1 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_subcompactions: 1 2023-07-14T19:21:40.459 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T19:21:40.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T19:21:40.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T19:21:40.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T19:21:40.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T19:21:40.460 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_open_files: -1 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T19:21:40.461 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_background_flushes: -1 2023-07-14T19:21:40.462 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Compression algorithms supported: 2023-07-14T19:21:40.462 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T19:21:40.462 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kZSTD supported: 0 2023-07-14T19:21:40.462 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kXpressCompression supported: 0 2023-07-14T19:21:40.462 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kZlibCompression supported: 1 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kSnappyCompression supported: 1 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kLZ4Compression supported: 1 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: kBZip2Compression supported: 0 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T19:21:40.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.merge_operator: 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_filter: None 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T19:21:40.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55865359f3a0) 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks: 1 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout: pin_top_level_index_and_filter: 1 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_type: 0 2023-07-14T19:21:40.465 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_index_type: 0 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_shortening: 1 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: checksum: 4 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: no_block_cache: 0 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache: 0x55865355f1f0 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_name: BinnedLRUCache 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_options: 2023-07-14T19:21:40.466 INFO:journalctl@ceph.mon.c.smithi161.stdout: capacity : 536870912 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_shard_bits : 4 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: strict_capacity_limit : 0 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: high_pri_pool_ratio: 0.000 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_cache_compressed: (nil) 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: persistent_cache: (nil) 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size: 4096 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_size_deviation: 10 2023-07-14T19:21:40.467 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_restart_interval: 16 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: index_block_restart_interval: 1 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: metadata_block_size: 4096 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: partition_filters: 0 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: use_delta_encoding: 1 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: filter_policy: bloomfilter 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: whole_key_filtering: 1 2023-07-14T19:21:40.468 INFO:journalctl@ceph.mon.c.smithi161.stdout: verify_compression: 0 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: read_amp_bytes_per_bit: 0 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: format_version: 5 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_index_compression: 1 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: block_align: 0 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: max_auto_readahead_size: 262144 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: prepopulate_block_cache: 0 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: initial_auto_readahead_size: 8192 2023-07-14T19:21:40.469 INFO:journalctl@ceph.mon.c.smithi161.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression: NoCompression 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T19:21:40.470 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.num_levels: 7 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T19:21:40.471 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T19:21:40.472 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T19:21:40.473 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T19:21:40.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T19:21:40.475 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T19:21:40.476 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.table_properties_collectors: 2023-07-14T19:21:40.477 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.inplace_update_support: 0 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.bloom_locality: 0 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.max_successive_merges: 0 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T19:21:40.478 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.ttl: 2592000 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.enable_blob_files: false 2023-07-14T19:21:40.479 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.min_blob_size: 0 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T19:21:40.480 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 798eebc8-6241-472a-a4c4-afeafd09d2dc 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362500149218, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362500149661, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689362500, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "798eebc8-6241-472a-a4c4-afeafd09d2dc", "db_session_id": "N4I4PEC5L51ASL23F6KO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T19:21:40.481 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362500149733, "job": 1, "event": "recovery_finished"} 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5586535f8000 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: DB pointer 0x5586535e4000 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** DB Stats ** 2023-07-14T19:21:40.482 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2023-07-14T19:21:40.483 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2023-07-14T19:21:40.484 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** Compaction Stats [default] ** 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T19:21:40.485 INFO:journalctl@ceph.mon.c.smithi161.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T19:21:40.486 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache BinnedLRUCache@0x55865355f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: using public_addr v2:172.21.15.161:0/0 -> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: starting mon.c rank -1 at public addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] at bind addrs [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:40.487 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(???) e0 preinit fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).mds e1 new map 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout: e1 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout: legacy client fscid: -1 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout: 2023-07-14T19:21:40.488 INFO:journalctl@ceph.mon.c.smithi161.stdout: No filesystems configured 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T19:21:40.489 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/983947571' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/983947571' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:40.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c;count:3 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:40.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: Deploying daemon mon.c on smithi161 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 ceph-mon[129184]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-14T19:21:40.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 bash[129117]: 3b9c95c08a93e7af6a567f5d0a0e76b061dad032a552b4a009a2f5c8be54c23c 2023-07-14T19:21:40.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:40 smithi161 systemd[1]: Started Ceph mon.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:21:41.496 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T19:21:41.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T19:21:42.065 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:21:45.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:45.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: mon.a calling monitor election 2023-07-14T19:21:45.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:45.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: mon.c calling monitor election 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T19:21:45.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: fsmap 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: mgrmap e13: x(active, since 40s) 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: overall HEALTH_OK 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:45.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:45.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:45.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:45.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: mon.a calling monitor election 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: mon.c calling monitor election 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:45.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: fsmap 2023-07-14T19:21:45.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: mgrmap e13: x(active, since 40s) 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: overall HEALTH_OK 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:45.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:46.398 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.396+0000 7fe9952de700 1 -- 172.21.15.161:0/1269941560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 msgr2=0x7fe978005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.396+0000 7fe9952de700 1 --2- 172.21.15.161:0/1269941560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe978005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9900ffc70 tx=0x7fe98000fe60 comp rx=0 tx=0).stop 2023-07-14T19:21:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.397+0000 7fe9952de700 1 -- 172.21.15.161:0/1269941560 shutdown_connections 2023-07-14T19:21:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.397+0000 7fe9952de700 1 --2- 172.21.15.161:0/1269941560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe978005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.397+0000 7fe9952de700 1 -- 172.21.15.161:0/1269941560 >> 172.21.15.161:0/1269941560 conn(0x7fe990074500 msgr2=0x7fe990074900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.397+0000 7fe9952de700 1 -- 172.21.15.161:0/1269941560 shutdown_connections 2023-07-14T19:21:46.399 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.397+0000 7fe9952de700 1 -- 172.21.15.161:0/1269941560 wait complete. 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.398+0000 7fe9952de700 1 Processor -- start 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.398+0000 7fe9952de700 1 -- start start 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.398+0000 7fe9952de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe9901003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.398+0000 7fe9952de700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 0x7fe990100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.398+0000 7fe9952de700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901023c0 con 0x7fe9780035c0 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.398+0000 7fe9952de700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901a0f00 con 0x7fe990100930 2023-07-14T19:21:46.401 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe9901003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:46.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98e7fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 0x7fe990100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:46.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe9901003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.161:45050/0 (socket says 172.21.15.161:45050) 2023-07-14T19:21:46.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 -- 172.21.15.161:0/169718197 learned_addr learned my addr 172.21.15.161:0/169718197 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:21:46.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98e7fc700 1 -- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 msgr2=0x7fe990100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-14T19:21:46.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98e7fc700 1 -- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 msgr2=0x7fe990100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T19:21:46.402 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98e7fc700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 0x7fe990100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T19:21:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98e7fc700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 0x7fe990100da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:21:46.403 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 -- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 msgr2=0x7fe990100da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:21:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 0x7fe990100da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 -- 172.21.15.161:0/169718197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98000f8b0 con 0x7fe9780035c0 2023-07-14T19:21:46.404 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe98effd700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe9901003f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe980013be0 tx=0x7fe980013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:46.406 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe980023070 con 0x7fe9780035c0 2023-07-14T19:21:46.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe980004bd0 con 0x7fe9780035c0 2023-07-14T19:21:46.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9901a1100 con 0x7fe9780035c0 2023-07-14T19:21:46.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.399+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fe980003ea0 con 0x7fe9780035c0 2023-07-14T19:21:46.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.400+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901a1620 con 0x7fe9780035c0 2023-07-14T19:21:46.407 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.400+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fe980004110 con 0x7fe9780035c0 2023-07-14T19:21:46.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.401+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990061c70 con 0x7fe9780035c0 2023-07-14T19:21:46.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.401+0000 7fe977fff700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe97c0385f0 0x7fe97c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:46.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.401+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe98004d7c0 con 0x7fe9780035c0 2023-07-14T19:21:46.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.401+0000 7fe98e7fc700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe97c0385f0 0x7fe97c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:46.408 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.401+0000 7fe98e7fc700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe97c0385f0 0x7fe97c03aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe984009850 tx=0x7fe984008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:46.409 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.406+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe98001a740 con 0x7fe9780035c0 2023-07-14T19:21:46.671 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.669+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe9901a1a70 con 0x7fe9780035c0 2023-07-14T19:21:46.672 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.670+0000 7fe977fff700 1 -- 172.21.15.161:0/169718197 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fe98001a380 con 0x7fe9780035c0 2023-07-14T19:21:46.672 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:21:46.672 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","modified":"2023-07-14T19:21:40.176134Z","created":"2023-07-14T19:20:10.680493Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T19:21:46.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe97c0385f0 msgr2=0x7fe97c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:46.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe97c0385f0 0x7fe97c03aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe984009850 tx=0x7fe984008040 comp rx=0 tx=0).stop 2023-07-14T19:21:46.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 msgr2=0x7fe9901003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:46.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe9901003f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe980013be0 tx=0x7fe980013cc0 comp rx=0 tx=0).stop 2023-07-14T19:21:46.674 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 shutdown_connections 2023-07-14T19:21:46.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe97c0385f0 0x7fe97c03aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:46.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9780035c0 0x7fe9901003f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:46.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 --2- 172.21.15.161:0/169718197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe990100930 0x7fe990100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:46.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 >> 172.21.15.161:0/169718197 conn(0x7fe990074500 msgr2=0x7fe990067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:46.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 shutdown_connections 2023-07-14T19:21:46.675 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:46.672+0000 7fe9952de700 1 -- 172.21.15.161:0/169718197 wait complete. 2023-07-14T19:21:46.676 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2023-07-14T19:21:46.694 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:46 smithi161 ceph-mon[129184]: Deploying daemon mon.b on smithi133 2023-07-14T19:21:46.695 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:46.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:46 smithi082 ceph-mon[127461]: Deploying daemon mon.b on smithi133 2023-07-14T19:21:46.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:47.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:47 smithi082 ceph-mon[127461]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:47.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:47 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/169718197' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T19:21:47.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:47 smithi161 ceph-mon[129184]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:47.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:47 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/169718197' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T19:21:48.097 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:47 smithi133 systemd[1]: Starting Ceph mon.b for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:21:48.127 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-14T19:21:48.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph mon dump -f json 2023-07-14T19:21:48.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 podman[128854]: 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 bash[128854]: 35049331e818910823c9843cc546522cd508d412fa11d4600a7c4e0d6dc93ee6 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: set uid:gid to 167:167 (ceph:ceph) 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev), process ceph-mon, pid 7 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pidfile_write: ignore empty --pid-file 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: load: jerasure load: lrc 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: RocksDB version: 7.9.2 2023-07-14T19:21:48.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Git sha 0 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Compile date 2023-07-14 14:45:42 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: DB SUMMARY 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: DB Session ID: 7UPA0QVBQ0HRTEC1UL31 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: CURRENT file: CURRENT 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: IDENTITY file: IDENTITY 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-14T19:21:48.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.error_if_exists: 0 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.create_if_missing: 0 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.paranoid_checks: 1 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-14T19:21:48.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.env: 0x5596ade98c60 2023-07-14T19:21:48.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.fs: PosixFileSystem 2023-07-14T19:21:48.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.info_log: 0x5596af7752a0 2023-07-14T19:21:48.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_file_opening_threads: 16 2023-07-14T19:21:48.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.statistics: (nil) 2023-07-14T19:21:48.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.use_fsync: 0 2023-07-14T19:21:48.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_log_file_size: 0 2023-07-14T19:21:48.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-14T19:21:48.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-14T19:21:48.440 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.keep_log_file_num: 1000 2023-07-14T19:21:48.440 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.recycle_log_file_num: 0 2023-07-14T19:21:48.440 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.allow_fallocate: 1 2023-07-14T19:21:48.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.allow_mmap_reads: 0 2023-07-14T19:21:48.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.allow_mmap_writes: 0 2023-07-14T19:21:48.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.use_direct_reads: 0 2023-07-14T19:21:48.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-14T19:21:48.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.create_missing_column_families: 0 2023-07-14T19:21:48.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.db_log_dir: 2023-07-14T19:21:48.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.wal_dir: 2023-07-14T19:21:48.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-14T19:21:48.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-14T19:21:48.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-14T19:21:48.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-14T19:21:48.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-14T19:21:48.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-14T19:21:48.447 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.advise_random_on_open: 1 2023-07-14T19:21:48.447 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.db_write_buffer_size: 0 2023-07-14T19:21:48.447 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.write_buffer_manager: 0x5596af6963c0 2023-07-14T19:21:48.447 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-14T19:21:48.448 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-14T19:21:48.448 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-14T19:21:48.448 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.rate_limiter: (nil) 2023-07-14T19:21:48.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-14T19:21:48.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.wal_recovery_mode: 2 2023-07-14T19:21:48.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.enable_thread_tracking: 0 2023-07-14T19:21:48.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.enable_pipelined_write: 0 2023-07-14T19:21:48.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.unordered_write: 0 2023-07-14T19:21:48.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.row_cache: None 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.wal_filter: None 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-14T19:21:48.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.allow_ingest_behind: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.two_write_queues: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.manual_wal_flush: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.wal_compression: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.atomic_flush: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-14T19:21:48.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.log_readahead_size: 0 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.best_efforts_recovery: 0 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.allow_data_in_errors: 0 2023-07-14T19:21:48.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.db_host_id: __hostname__ 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_background_jobs: 2 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_background_compactions: -1 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_subcompactions: 1 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-14T19:21:48.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_total_wal_size: 0 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_open_files: -1 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bytes_per_sync: 0 2023-07-14T19:21:48.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_readahead_size: 0 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_background_flushes: -1 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Compression algorithms supported: 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kZSTD supported: 0 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kXpressCompression supported: 0 2023-07-14T19:21:48.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kLZ4HCCompression supported: 1 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kZlibCompression supported: 1 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kSnappyCompression supported: 1 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kLZ4Compression supported: 1 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: kBZip2Compression supported: 0 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-14T19:21:48.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.merge_operator: 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_filter: None 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_filter_factory: None 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.sst_partitioner_factory: None 2023-07-14T19:21:48.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5596af7753a0) 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout: cache_index_and_filter_blocks: 1 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout: pin_top_level_index_and_filter: 1 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout: index_type: 0 2023-07-14T19:21:48.458 INFO:journalctl@ceph.mon.b.smithi133.stdout: data_block_index_type: 0 2023-07-14T19:21:48.459 INFO:journalctl@ceph.mon.b.smithi133.stdout: index_shortening: 1 2023-07-14T19:21:48.459 INFO:journalctl@ceph.mon.b.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-14T19:21:48.459 INFO:journalctl@ceph.mon.b.smithi133.stdout: checksum: 4 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: no_block_cache: 0 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache: 0x5596af7351f0 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache_name: BinnedLRUCache 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache_options: 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: capacity : 536870912 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: num_shard_bits : 4 2023-07-14T19:21:48.460 INFO:journalctl@ceph.mon.b.smithi133.stdout: strict_capacity_limit : 0 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: high_pri_pool_ratio: 0.000 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_cache_compressed: (nil) 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: persistent_cache: (nil) 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_size: 4096 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_size_deviation: 10 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_restart_interval: 16 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: index_block_restart_interval: 1 2023-07-14T19:21:48.461 INFO:journalctl@ceph.mon.b.smithi133.stdout: metadata_block_size: 4096 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: partition_filters: 0 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: use_delta_encoding: 1 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: filter_policy: bloomfilter 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: whole_key_filtering: 1 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: verify_compression: 0 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: read_amp_bytes_per_bit: 0 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: format_version: 5 2023-07-14T19:21:48.462 INFO:journalctl@ceph.mon.b.smithi133.stdout: enable_index_compression: 1 2023-07-14T19:21:48.463 INFO:journalctl@ceph.mon.b.smithi133.stdout: block_align: 0 2023-07-14T19:21:48.463 INFO:journalctl@ceph.mon.b.smithi133.stdout: max_auto_readahead_size: 262144 2023-07-14T19:21:48.463 INFO:journalctl@ceph.mon.b.smithi133.stdout: prepopulate_block_cache: 0 2023-07-14T19:21:48.463 INFO:journalctl@ceph.mon.b.smithi133.stdout: initial_auto_readahead_size: 8192 2023-07-14T19:21:48.464 INFO:journalctl@ceph.mon.b.smithi133.stdout: num_file_reads_for_auto_readahead: 2 2023-07-14T19:21:48.464 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.write_buffer_size: 33554432 2023-07-14T19:21:48.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_write_buffer_number: 2 2023-07-14T19:21:48.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression: NoCompression 2023-07-14T19:21:48.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression: Disabled 2023-07-14T19:21:48.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.prefix_extractor: nullptr 2023-07-14T19:21:48.465 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.num_levels: 7 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-14T19:21:48.466 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-14T19:21:48.467 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-14T19:21:48.467 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-14T19:21:48.467 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-14T19:21:48.467 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-14T19:21:48.467 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-14T19:21:48.467 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-14T19:21:48.468 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-14T19:21:48.468 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.level: 32767 2023-07-14T19:21:48.468 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.strategy: 0 2023-07-14T19:21:48.468 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-14T19:21:48.468 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-14T19:21:48.469 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-14T19:21:48.469 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-14T19:21:48.469 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.enabled: false 2023-07-14T19:21:48.469 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-14T19:21:48.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-14T19:21:48.470 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-14T19:21:48.471 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-14T19:21:48.472 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.target_file_size_base: 67108864 2023-07-14T19:21:48.472 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-14T19:21:48.472 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-14T19:21:48.472 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-14T19:21:48.473 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-14T19:21:48.473 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-14T19:21:48.473 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-14T19:21:48.474 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-14T19:21:48.474 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-14T19:21:48.474 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-14T19:21:48.474 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-14T19:21:48.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-14T19:21:48.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-14T19:21:48.475 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-14T19:21:48.476 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-14T19:21:48.476 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.arena_block_size: 1048576 2023-07-14T19:21:48.476 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-14T19:21:48.476 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-14T19:21:48.477 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.disable_auto_compactions: 0 2023-07-14T19:21:48.477 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-14T19:21:48.477 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-14T19:21:48.478 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-14T19:21:48.478 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-14T19:21:48.478 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-14T19:21:48.478 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-14T19:21:48.479 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-14T19:21:48.479 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-14T19:21:48.479 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-14T19:21:48.479 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-14T19:21:48.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.table_properties_collectors: 2023-07-14T19:21:48.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.inplace_update_support: 0 2023-07-14T19:21:48.480 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-14T19:21:48.481 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-14T19:21:48.481 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-14T19:21:48.481 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-14T19:21:48.482 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.bloom_locality: 0 2023-07-14T19:21:48.482 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.max_successive_merges: 0 2023-07-14T19:21:48.482 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-14T19:21:48.482 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.paranoid_file_checks: 0 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.force_consistency_checks: 1 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.report_bg_io_stats: 0 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.ttl: 2592000 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-14T19:21:48.483 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.enable_blob_files: false 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.min_blob_size: 0 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.blob_file_size: 268435456 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-14T19:21:48.484 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.blob_file_starting_level: 0 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fe67b247-5664-4486-9be8-034dc1303594 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362508277044, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-14T19:21:48.485 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362508277453, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689362508, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fe67b247-5664-4486-9be8-034dc1303594", "db_session_id": "7UPA0QVBQ0HRTEC1UL31", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689362508277519, "job": 1, "event": "recovery_finished"} 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5596af7ce000 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: DB pointer 0x5596af7ba000 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-14T19:21:48.486 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** DB Stats ** 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-14T19:21:48.487 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** Compaction Stats [default] ** 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:48.488 INFO:journalctl@ceph.mon.b.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** Compaction Stats [default] ** 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-14T19:21:48.489 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T19:21:48.490 INFO:journalctl@ceph.mon.b.smithi133.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: Block cache BinnedLRUCache@0x5596af7351f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-14T19:21:48.491 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: using public_addr v2:172.21.15.133:0/0 -> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: starting mon.b rank -1 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(???) e0 preinit fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 systemd[1]: Started Ceph mon.b for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).mds e1 new map 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: e1 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T19:21:48.492 INFO:journalctl@ceph.mon.b.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout: legacy client fscid: -1 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout: 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout: No filesystems configured 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-14T19:21:48.493 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:21:48.494 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/983947571' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/983947571' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.495 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:48.496 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Saving service mon spec with placement smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c;count:3 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:48.497 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Deploying daemon mon.c on smithi161 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.498 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.a calling monitor election 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.c calling monitor election 2023-07-14T19:21:48.499 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: monmap e2: 2 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:48.500 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: fsmap 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mgrmap e13: x(active, since 40s) 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: overall HEALTH_OK 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:48.501 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: Deploying daemon mon.b on smithi133 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/169718197' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expand map: {default=false} 2023-07-14T19:21:48.502 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta from 'false' to 'false' 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expanded map: {default=false} 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expand map: {default=info} 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta from 'info' to 'info' 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expanded map: {default=info} 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expand map: {default=daemon} 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expanded map: {default=daemon} 2023-07-14T19:21:48.503 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expand map: {default=debug} 2023-07-14T19:21:48.504 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta from 'debug' to 'debug' 2023-07-14T19:21:48.504 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:48 smithi133 ceph-mon[128894]: expand_channel_meta expanded map: {default=debug} 2023-07-14T19:21:48.691 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:21:53.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.629+0000 7efc0f50b700 1 -- 172.21.15.161:0/2416044806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efbf40035c0 msgr2=0x7efbf4005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:53.632 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.629+0000 7efc0f50b700 1 --2- 172.21.15.161:0/2416044806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efbf40035c0 0x7efbf4005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efc080fe760 tx=0x7efbf8014e70 comp rx=0 tx=0).stop 2023-07-14T19:21:53.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.629+0000 7efc0f50b700 1 -- 172.21.15.161:0/2416044806 shutdown_connections 2023-07-14T19:21:53.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.629+0000 7efc0f50b700 1 --2- 172.21.15.161:0/2416044806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efbf40035c0 0x7efbf4005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.633 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.629+0000 7efc0f50b700 1 -- 172.21.15.161:0/2416044806 >> 172.21.15.161:0/2416044806 conn(0x7efc08074500 msgr2=0x7efc08074900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:53.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.630+0000 7efc0f50b700 1 -- 172.21.15.161:0/2416044806 shutdown_connections 2023-07-14T19:21:53.634 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.630+0000 7efc0f50b700 1 -- 172.21.15.161:0/2416044806 wait complete. 2023-07-14T19:21:53.638 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 Processor -- start 2023-07-14T19:21:53.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 -- start start 2023-07-14T19:21:53.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:53.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc081952a0 0x7efc08199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:53.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 0x7efc0819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:53.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08195970 con 0x7efc081952a0 2023-07-14T19:21:53.639 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081a19c0 con 0x7efc08199c50 2023-07-14T19:21:53.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0f50b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc081a1b00 con 0x7efbf40035c0 2023-07-14T19:21:53.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:53.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60088/0 (socket says 172.21.15.161:60088) 2023-07-14T19:21:53.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 -- 172.21.15.161:0/1163151484 learned_addr learned my addr 172.21.15.161:0/1163151484 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:21:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0caa6700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc081952a0 0x7efc08199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 msgr2=0x7efc08194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-14T19:21:53.641 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 msgr2=0x7efc08194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T19:21:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T19:21:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0d2a7700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:21:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0daa8700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 0x7efc0819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0daa8700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 msgr2=0x7efc0819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-14T19:21:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0daa8700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 msgr2=0x7efc0819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-14T19:21:53.642 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0daa8700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 0x7efc0819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-14T19:21:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0caa6700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 msgr2=0x7efc08194d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:21:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0caa6700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.643 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.631+0000 7efc0caa6700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 msgr2=0x7efc0819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:53.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efc0daa8700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 0x7efc0819a0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:21:53.644 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efc0caa6700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 0x7efc0819a0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efc0caa6700 1 -- 172.21.15.161:0/1163151484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80148c0 con 0x7efc081952a0 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efc0caa6700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc081952a0 0x7efc08199710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc0400b580 tx=0x7efc0400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04004230 con 0x7efc081952a0 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc04004390 con 0x7efc081952a0 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc081a1db0 con 0x7efc081952a0 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04011a70 con 0x7efc081952a0 2023-07-14T19:21:53.645 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.632+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc081a2280 con 0x7efc081952a0 2023-07-14T19:21:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.633+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7efc04011cb0 con 0x7efc081952a0 2023-07-14T19:21:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.633+0000 7efbfe7fc700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efbec038c80 0x7efbec03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.633+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc08061c70 con 0x7efc081952a0 2023-07-14T19:21:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.633+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc0404ad50 con 0x7efc081952a0 2023-07-14T19:21:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.633+0000 7efc0d2a7700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efbec038c80 0x7efbec03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:53.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.634+0000 7efc0d2a7700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efbec038c80 0x7efbec03b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efbf800f470 tx=0x7efbf800f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:53.647 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.638+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efc0401a400 con 0x7efc081952a0 2023-07-14T19:21:53.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:53.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: mon.a calling monitor election 2023-07-14T19:21:53.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: mon.c calling monitor election 2023-07-14T19:21:53.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:53.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:53.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T19:21:53.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:53.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: fsmap 2023-07-14T19:21:53.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:53.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: mgrmap e13: x(active, since 48s) 2023-07-14T19:21:53.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T19:21:53.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T19:21:53.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T19:21:53.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: mon.b (rank 2) addr [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] is down (out of quorum) 2023-07-14T19:21:53.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:53.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: mon.a calling monitor election 2023-07-14T19:21:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: mon.c calling monitor election 2023-07-14T19:21:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: fsmap 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: mgrmap e13: x(active, since 48s) 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T19:21:53.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T19:21:53.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T19:21:53.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: mon.b (rank 2) addr [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] is down (out of quorum) 2023-07-14T19:21:53.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:53.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:53.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.928+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efc081960c0 con 0x7efc081952a0 2023-07-14T19:21:53.932 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.928+0000 7efbfe7fc700 1 -- 172.21.15.161:0/1163151484 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7efc0400f2e0 con 0x7efc081952a0 2023-07-14T19:21:53.932 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-07-14T19:21:53.933 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","modified":"2023-07-14T19:21:48.309876Z","created":"2023-07-14T19:20:10.680493Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-14T19:21:53.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efbec038c80 msgr2=0x7efbec03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:53.933 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efbec038c80 0x7efbec03b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efbf800f470 tx=0x7efbf800f3c0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc081952a0 msgr2=0x7efc08199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc081952a0 0x7efc08199710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efc0400b580 tx=0x7efc0400b940 comp rx=0 tx=0).stop 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 shutdown_connections 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc08199c50 0x7efc0819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efbf40035c0 0x7efc08194d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efbec038c80 0x7efbec03b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.934 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 --2- 172.21.15.161:0/1163151484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc081952a0 0x7efc08199710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:53.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 >> 172.21.15.161:0/1163151484 conn(0x7efc08074500 msgr2=0x7efc08102630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:53.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 shutdown_connections 2023-07-14T19:21:53.935 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:53.931+0000 7efc0f50b700 1 -- 172.21.15.161:0/1163151484 wait complete. 2023-07-14T19:21:53.935 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2023-07-14T19:21:54.364 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-14T19:21:54.365 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph config generate-minimal-conf 2023-07-14T19:21:54.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:54.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:54.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:54.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:54 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/1163151484' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T19:21:54.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:54.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:54.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:54.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:54.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:54 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/1163151484' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T19:21:54.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:54.930 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:21:55.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:21:55.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: mon.a calling monitor election 2023-07-14T19:21:55.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: mon.c calling monitor election 2023-07-14T19:21:55.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:55.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:21:55.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:55.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:55.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:55.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:55.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:55.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:55.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:55.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:55.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-14T19:21:55.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:55.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: fsmap 2023-07-14T19:21:55.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: mgrmap e13: x(active, since 48s) 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: mon.b (rank 2) addr [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] is down (out of quorum) 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:55.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:55.691 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:55.691 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:55.691 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:21:55.691 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/1163151484' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-14T19:21:55.691 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 -- 172.21.15.82:0/2439995088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 msgr2=0x7f9a28104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/2439995088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a28104a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009910 tx=0x7f9a1c009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:56.071 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 -- 172.21.15.82:0/2439995088 shutdown_connections 2023-07-14T19:21:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/2439995088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a28104a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 -- 172.21.15.82:0/2439995088 >> 172.21.15.82:0/2439995088 conn(0x7f9a280743e0 msgr2=0x7f9a280747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 -- 172.21.15.82:0/2439995088 shutdown_connections 2023-07-14T19:21:56.072 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.070+0000 7f9a2ca72700 1 -- 172.21.15.82:0/2439995088 wait complete. 2023-07-14T19:21:56.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 Processor -- start 2023-07-14T19:21:56.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 -- start start 2023-07-14T19:21:56.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a2818df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:56.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2818e4b0 0x7f9a2818e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:56.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a281a5460 0x7f9a281a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:56.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a281a7d80 con 0x7f9a28104660 2023-07-14T19:21:56.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a281a7ec0 con 0x7f9a281a5460 2023-07-14T19:21:56.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2ca72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a281a8050 con 0x7f9a2818e4b0 2023-07-14T19:21:56.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a2818df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:56.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a2818df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39832/0 (socket says 172.21.15.82:39832) 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 -- 172.21.15.82:0/142775958 learned_addr learned my addr 172.21.15.82:0/142775958 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a26d9d700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a281a5460 0x7f9a281a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 -- 172.21.15.82:0/142775958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2818e4b0 msgr2=0x7f9a2818e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2818e4b0 0x7f9a2818e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 -- 172.21.15.82:0/142775958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a281a5460 msgr2=0x7f9a281a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a281a5460 0x7f9a281a7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.071+0000 7f9a2659c700 1 -- 172.21.15.82:0/142775958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c0095f0 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.072+0000 7f9a2659c700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a2818df70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c004ce0 tx=0x7f9a1c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.072+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c01d070 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.072+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a281a82d0 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.072+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c00be20 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.072+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00f460 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.072+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a281a8820 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.073+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f9a1c022670 con 0x7f9a28104660 2023-07-14T19:21:56.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.073+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28103010 con 0x7f9a28104660 2023-07-14T19:21:56.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.073+0000 7f9a177fe700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9a10038c30 0x7f9a1003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:56.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.073+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a1c04c780 con 0x7f9a28104660 2023-07-14T19:21:56.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.073+0000 7f9a25d9b700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9a10038c30 0x7f9a1003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:56.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.074+0000 7f9a25d9b700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9a10038c30 0x7f9a1003b0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9a180098e0 tx=0x7f9a18006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:56.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.078+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9a1c00ee10 con 0x7f9a28104660 2023-07-14T19:21:56.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.283+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9a281029f0 con 0x7f9a28104660 2023-07-14T19:21:56.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.284+0000 7f9a177fe700 1 -- 172.21.15.82:0/142775958 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f9a1c027070 con 0x7f9a28104660 2023-07-14T19:21:56.285 INFO:teuthology.orchestra.run.smithi082.stdout:# minimal ceph.conf for 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:56.285 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2023-07-14T19:21:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:21:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: mon_host = [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2023-07-14T19:21:56.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9a10038c30 msgr2=0x7f9a1003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:56.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9a10038c30 0x7f9a1003b0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9a180098e0 tx=0x7f9a18006dd0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 msgr2=0x7f9a2818df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:56.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a2818df70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c004ce0 tx=0x7f9a1c004dc0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 shutdown_connections 2023-07-14T19:21:56.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a281a5460 0x7f9a281a7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9a10038c30 0x7f9a1003b0e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a28104660 0x7f9a2818df70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 --2- 172.21.15.82:0/142775958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2818e4b0 0x7f9a2818e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 >> 172.21.15.82:0/142775958 conn(0x7f9a280743e0 msgr2=0x7f9a28100150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 shutdown_connections 2023-07-14T19:21:56.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:21:56.286+0000 7f9a2ca72700 1 -- 172.21.15.82:0/142775958 wait complete. 2023-07-14T19:21:56.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: mon.b calling monitor election 2023-07-14T19:21:56.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:21:56.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:21:56.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:21:56.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: mon.b calling monitor election 2023-07-14T19:21:56.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: mon.c calling monitor election 2023-07-14T19:21:56.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: mon.a calling monitor election 2023-07-14T19:21:56.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T19:21:56.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:56.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: fsmap 2023-07-14T19:21:56.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:56.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: mgrmap e13: x(active, since 50s) 2023-07-14T19:21:56.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T19:21:56.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Cluster is now healthy 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: overall HEALTH_OK 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T19:21:56.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:56.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:56.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:56.439 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:56 smithi133 ceph-mon[128894]: Reconfiguring daemon mon.a on smithi082 2023-07-14T19:21:56.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: mon.b calling monitor election 2023-07-14T19:21:56.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:21:56.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:21:56.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:21:56.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: mon.b calling monitor election 2023-07-14T19:21:56.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: mon.c calling monitor election 2023-07-14T19:21:56.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: mon.a calling monitor election 2023-07-14T19:21:56.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T19:21:56.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:56.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: fsmap 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: mgrmap e13: x(active, since 50s) 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Cluster is now healthy 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: overall HEALTH_OK 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T19:21:56.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:56.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:56.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:56.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:56 smithi082 ceph-mon[127461]: Reconfiguring daemon mon.a on smithi082 2023-07-14T19:21:56.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: mon.b calling monitor election 2023-07-14T19:21:56.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:21:56.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: mon.b calling monitor election 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: mon.c calling monitor election 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: mon.a calling monitor election 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: monmap e3: 3 mons at {a=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],b=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0]} removed_ranks: {} 2023-07-14T19:21:56.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: fsmap 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: osdmap e4: 0 total, 0 up, 0 in 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: mgrmap e13: x(active, since 50s) 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Cluster is now healthy 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: overall HEALTH_OK 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:56.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Reconfiguring mon.a (unknown last config time)... 2023-07-14T19:21:56.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:56.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:56.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:56.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:56 smithi161 ceph-mon[129184]: Reconfiguring daemon mon.a on smithi082 2023-07-14T19:21:56.780 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-14T19:21:56.781 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:56.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T19:21:56.812 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:56.813 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:56.878 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:21:56.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T19:21:56.916 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:21:56.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:56.989 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:21:56.989 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2023-07-14T19:21:57.027 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:21:57.027 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:21:57.097 INFO:tasks.cephadm:Adding mgr.x on smithi082 2023-07-14T19:21:57.098 INFO:tasks.cephadm:Adding mgr.y on smithi133 2023-07-14T19:21:57.099 INFO:tasks.cephadm:Adding mgr.z on smithi161 2023-07-14T19:21:57.099 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch apply mgr '3;smithi082=x;smithi133=y;smithi161=z' 2023-07-14T19:21:57.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:57 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/142775958' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:57.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:57.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:57 smithi133 ceph-mon[128894]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:57.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:57 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/142775958' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:57.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:57.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:57 smithi082 ceph-mon[127461]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:57.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:57 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/142775958' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:57.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:21:57.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:57 smithi161 ceph-mon[129184]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:57.693 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:21:58.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:58.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:58.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: Reconfiguring mon.b (monmap changed)... 2023-07-14T19:21:58.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:58.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:58.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:58.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:58 smithi133 ceph-mon[128894]: Reconfiguring daemon mon.b on smithi133 2023-07-14T19:21:58.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: Reconfiguring mon.b (monmap changed)... 2023-07-14T19:21:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:58 smithi082 ceph-mon[127461]: Reconfiguring daemon mon.b on smithi133 2023-07-14T19:21:58.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:58.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:58.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: Reconfiguring mon.b (monmap changed)... 2023-07-14T19:21:58.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:58.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:58.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:58.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:58 smithi161 ceph-mon[129184]: Reconfiguring daemon mon.b on smithi133 2023-07-14T19:21:59.045 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 -- 172.21.15.161:0/2406160115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52000fde70 msgr2=0x7f52000fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 --2- 172.21.15.161:0/2406160115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52000fde70 0x7f52000fe280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009910 tx=0x7f51f0009c20 comp rx=0 tx=0).stop 2023-07-14T19:21:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 -- 172.21.15.161:0/2406160115 shutdown_connections 2023-07-14T19:21:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 --2- 172.21.15.161:0/2406160115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52000fde70 0x7f52000fe280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 -- 172.21.15.161:0/2406160115 >> 172.21.15.161:0/2406160115 conn(0x7f52000f9450 msgr2=0x7f52000fb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 -- 172.21.15.161:0/2406160115 shutdown_connections 2023-07-14T19:21:59.046 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 -- 172.21.15.161:0/2406160115 wait complete. 2023-07-14T19:21:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 Processor -- start 2023-07-14T19:21:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.044+0000 7f5205e69700 1 -- start start 2023-07-14T19:21:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 0x7f5200194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52001950b0 0x7f520019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5200195520 0x7f520019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:59.048 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019ce30 con 0x7f5200195520 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019cfa0 con 0x7f52001950b0 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019d160 con 0x7f52000fde70 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 0x7f5200194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 0x7f5200194b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60098/0 (socket says 172.21.15.161:60098) 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 -- 172.21.15.161:0/3512756394 learned_addr learned my addr 172.21.15.161:0/3512756394 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51fffff700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5200195520 0x7f520019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:59.049 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51feffd700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52001950b0 0x7f520019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 -- 172.21.15.161:0/3512756394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52001950b0 msgr2=0x7f520019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52001950b0 0x7f520019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 -- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5200195520 msgr2=0x7f520019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5200195520 0x7f520019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 -- 172.21.15.161:0/3512756394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00095f0 con 0x7f52000fde70 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51fffff700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5200195520 0x7f520019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51ff7fe700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 0x7f5200194b70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51f0000c00 tx=0x7f51f0004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:59.050 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f001d070 con 0x7f52000fde70 2023-07-14T19:21:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f000be20 con 0x7f52000fde70 2023-07-14T19:21:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520019d3e0 con 0x7f52000fde70 2023-07-14T19:21:59.051 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0022d40 con 0x7f52000fde70 2023-07-14T19:21:59.052 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.045+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520019d900 con 0x7f52000fde70 2023-07-14T19:21:59.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.046+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520018ec00 con 0x7f52000fde70 2023-07-14T19:21:59.056 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.047+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f51f0022470 con 0x7f52000fde70 2023-07-14T19:21:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.047+0000 7f51fcff9700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f51e8038c80 0x7f51e803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:21:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.047+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51f004cfb0 con 0x7f52000fde70 2023-07-14T19:21:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.047+0000 7f51feffd700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f51e8038c80 0x7f51e803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:21:59.057 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.048+0000 7f51feffd700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f51e8038c80 0x7f51e803b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009850 tx=0x7f51f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:21:59.058 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.052+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f51f0079780 con 0x7f52000fde70 2023-07-14T19:21:59.268 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.267+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi133=y;smithi161=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f5200060b10 con 0x7f51e8038c80 2023-07-14T19:21:59.275 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.273+0000 7f51fcff9700 1 -- 172.21.15.161:0/3512756394 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5200060b10 con 0x7f51e8038c80 2023-07-14T19:21:59.275 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2023-07-14T19:21:59.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f51e8038c80 msgr2=0x7f51e803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:59.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f51e8038c80 0x7f51e803b130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009850 tx=0x7f51f4008040 comp rx=0 tx=0).stop 2023-07-14T19:21:59.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 msgr2=0x7f5200194b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:21:59.284 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 0x7f5200194b70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51f0000c00 tx=0x7f51f0004cb0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 shutdown_connections 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f51e8038c80 0x7f51e803b130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5200195520 0x7f520019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52000fde70 0x7f5200194b70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 --2- 172.21.15.161:0/3512756394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52001950b0 0x7f520019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.275+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 >> 172.21.15.161:0/3512756394 conn(0x7f52000f9450 msgr2=0x7f52000fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:21:59.285 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.276+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 shutdown_connections 2023-07-14T19:21:59.286 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:21:59.276+0000 7f5205e69700 1 -- 172.21.15.161:0/3512756394 wait complete. 2023-07-14T19:21:59.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:59.490 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:59.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: Reconfiguring mon.c (monmap changed)... 2023-07-14T19:21:59.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:59.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:59.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:59.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: Reconfiguring daemon mon.c on smithi161 2023-07-14T19:21:59.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:21:59 smithi161 ceph-mon[129184]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:59.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:59.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:59.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: Reconfiguring mon.c (monmap changed)... 2023-07-14T19:21:59.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:59.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:59.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:59.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: Reconfiguring daemon mon.c on smithi161 2023-07-14T19:21:59.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:21:59 smithi133 ceph-mon[128894]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:59.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:59.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:21:59.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: Reconfiguring mon.c (monmap changed)... 2023-07-14T19:21:59.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-14T19:21:59.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-14T19:21:59.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:21:59.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: Reconfiguring daemon mon.c on smithi161 2023-07-14T19:21:59.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:21:59 smithi082 ceph-mon[127461]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:21:59.846 DEBUG:teuthology.orchestra.run.smithi133:mgr.y> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.y.service 2023-07-14T19:21:59.850 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.z.service 2023-07-14T19:21:59.852 INFO:tasks.cephadm:Deploying OSDs... 2023-07-14T19:21:59.853 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:21:59.853 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T19:21:59.874 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T19:21:59.874 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2023-07-14T19:21:59.882 INFO:journalctl@ceph.mgr.z.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:21:59.883 INFO:journalctl@ceph.mgr.y.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:21:59.934 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T19:21:59.934 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:21:59.934 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 377236 Links: 1 2023-07-14T19:21:59.935 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:21:59.935 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:21:59.935 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-07-14 19:21:14.380857878 +0000 2023-07-14T19:21:59.935 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-07-14 19:11:20.325686343 +0000 2023-07-14T19:21:59.935 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-07-14 19:11:20.325686343 +0000 2023-07-14T19:21:59.935 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-07-14T19:21:59.935 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T19:22:00.005 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-07-14T19:22:00.005 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-07-14T19:22:00.005 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000261104 s, 2.0 MB/s 2023-07-14T19:22:00.006 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T19:22:00.065 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2023-07-14T19:22:00.124 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T19:22:00.124 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:00.124 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 380328 Links: 1 2023-07-14T19:22:00.124 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:00.124 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:00.125 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-07-14 19:21:14.380857878 +0000 2023-07-14T19:22:00.125 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-07-14 19:11:20.817676706 +0000 2023-07-14T19:22:00.125 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-07-14 19:11:20.817676706 +0000 2023-07-14T19:22:00.125 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-07-14T19:22:00.125 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T19:22:00.190 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-07-14T19:22:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-07-14T19:22:00.191 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000228808 s, 2.2 MB/s 2023-07-14T19:22:00.193 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T19:22:00.252 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2023-07-14T19:22:00.310 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T19:22:00.311 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:00.311 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 380443 Links: 1 2023-07-14T19:22:00.311 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:00.311 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:00.311 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-07-14 19:21:14.380857878 +0000 2023-07-14T19:22:00.311 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-07-14 19:11:21.340666461 +0000 2023-07-14T19:22:00.312 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-07-14 19:11:21.340666461 +0000 2023-07-14T19:22:00.312 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-07-14T19:22:00.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T19:22:00.376 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-07-14T19:22:00.376 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-07-14T19:22:00.376 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000210603 s, 2.4 MB/s 2023-07-14T19:22:00.377 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T19:22:00.435 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2023-07-14T19:22:00.491 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T19:22:00.491 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:00.492 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 381324 Links: 1 2023-07-14T19:22:00.492 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:00.492 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:00.492 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-07-14 19:21:14.380857878 +0000 2023-07-14T19:22:00.492 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-07-14 19:11:21.857656334 +0000 2023-07-14T19:22:00.492 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-07-14 19:11:21.857656334 +0000 2023-07-14T19:22:00.493 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-07-14T19:22:00.493 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T19:22:00.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi133=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:00.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: Saving service mgr spec with placement smithi082=x;smithi133=y;smithi161=z;count:3 2023-07-14T19:22:00.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.555 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:00.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:00.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:00.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.556 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:00.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T19:22:00.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:00.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:00.557 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:00 smithi082 ceph-mon[127461]: Deploying daemon mgr.z on smithi161 2023-07-14T19:22:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-07-14T19:22:00.558 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-07-14T19:22:00.559 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000222533 s, 2.3 MB/s 2023-07-14T19:22:00.559 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T19:22:00.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi133=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:00.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: Saving service mgr spec with placement smithi082=x;smithi133=y;smithi161=z;count:3 2023-07-14T19:22:00.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.590 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T19:22:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:00 smithi161 ceph-mon[129184]: Deploying daemon mgr.z on smithi161 2023-07-14T19:22:00.618 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-07-14T19:22:00.618 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T19:22:00.640 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T19:22:00.641 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2023-07-14T19:22:00.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi082=x;smithi133=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:00.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: Saving service mgr spec with placement smithi082=x;smithi133=y;smithi161=z;count:3 2023-07-14T19:22:00.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:00.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:00.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:00.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:00.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:00.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T19:22:00.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:00.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:00.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:00 smithi133 ceph-mon[128894]: Deploying daemon mgr.z on smithi161 2023-07-14T19:22:00.700 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T19:22:00.700 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 381973 Links: 1 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 19:21:47.756371568 +0000 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 19:07:33.549092837 +0000 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 19:07:33.549092837 +0000 2023-07-14T19:22:00.701 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-07-14T19:22:00.702 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T19:22:00.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T19:22:00.769 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T19:22:00.769 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000254809 s, 2.0 MB/s 2023-07-14T19:22:00.770 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T19:22:00.830 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2023-07-14T19:22:00.890 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T19:22:00.890 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:00.890 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 381031 Links: 1 2023-07-14T19:22:00.890 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:00.890 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:00.891 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 19:21:47.756371568 +0000 2023-07-14T19:22:00.891 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 19:07:34.057082985 +0000 2023-07-14T19:22:00.891 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 19:07:34.057082985 +0000 2023-07-14T19:22:00.891 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-07-14T19:22:00.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T19:22:00.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T19:22:00.960 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T19:22:00.960 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000259461 s, 2.0 MB/s 2023-07-14T19:22:00.960 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T19:22:01.018 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2023-07-14T19:22:01.079 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T19:22:01.079 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:01.079 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 377267 Links: 1 2023-07-14T19:22:01.079 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:01.080 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:01.080 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 19:21:47.756371568 +0000 2023-07-14T19:22:01.080 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 19:07:34.553073366 +0000 2023-07-14T19:22:01.080 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 19:07:34.553073366 +0000 2023-07-14T19:22:01.080 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-07-14T19:22:01.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T19:22:01.146 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T19:22:01.147 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T19:22:01.147 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000228663 s, 2.2 MB/s 2023-07-14T19:22:01.147 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T19:22:01.204 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2023-07-14T19:22:01.262 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T19:22:01.262 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 376640 Links: 1 2023-07-14T19:22:01.262 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:01.263 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:01.263 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-07-14 19:21:47.756371568 +0000 2023-07-14T19:22:01.263 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-07-14 19:07:35.053063669 +0000 2023-07-14T19:22:01.263 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-07-14 19:07:35.053063669 +0000 2023-07-14T19:22:01.263 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2023-07-14T19:22:01.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T19:22:01.328 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-07-14T19:22:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-07-14T19:22:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000189007 s, 2.7 MB/s 2023-07-14T19:22:01.329 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T19:22:01.386 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-07-14T19:22:01.387 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2023-07-14T19:22:01.406 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-14T19:22:01.407 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2023-07-14T19:22:01.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:01 smithi133 ceph-mon[128894]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:01.447 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:01 smithi161 ceph-mon[129184]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:01.448 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 systemd[1]: Starting Ceph mgr.z for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:22:01.464 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-14T19:22:01.465 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:01.465 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 372355 Links: 1 2023-07-14T19:22:01.465 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:01.465 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:01.465 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 19:22:01.101459770 +0000 2023-07-14T19:22:01.466 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 19:08:10.879870012 +0000 2023-07-14T19:22:01.466 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 19:08:10.879870012 +0000 2023-07-14T19:22:01.466 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T19:22:01.466 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-14T19:22:01.546 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T19:22:01.547 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T19:22:01.547 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000221311 s, 2.3 MB/s 2023-07-14T19:22:01.549 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-14T19:22:01.607 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2023-07-14T19:22:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-14T19:22:01.663 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 378394 Links: 1 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 19:22:01.101459770 +0000 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 19:08:11.395858729 +0000 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 19:08:11.395858729 +0000 2023-07-14T19:22:01.664 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T19:22:01.665 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-14T19:22:01.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:01 smithi082 ceph-mon[127461]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:01.719 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 podman[131028]: 2023-07-14T19:22:01.732 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T19:22:01.732 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T19:22:01.732 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000225091 s, 2.3 MB/s 2023-07-14T19:22:01.733 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-14T19:22:01.794 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2023-07-14T19:22:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-14T19:22:01.853 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:01.853 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 380764 Links: 1 2023-07-14T19:22:01.854 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:01.854 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:01.854 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 19:22:01.101459770 +0000 2023-07-14T19:22:01.854 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 19:08:11.896847775 +0000 2023-07-14T19:22:01.854 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 19:08:11.896847775 +0000 2023-07-14T19:22:01.854 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T19:22:01.855 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-14T19:22:01.919 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T19:22:01.920 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T19:22:01.920 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000244604 s, 2.1 MB/s 2023-07-14T19:22:01.921 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-14T19:22:01.980 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2023-07-14T19:22:01.991 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 ceph-mgr[131168]: -- 172.21.15.161:0/4202254777 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce71ba51e0 con 0x55ce71b7a800 2023-07-14T19:22:01.992 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 bash[131028]: 60c177910ff11d02f6df0a671af0531fe10e9c8abd5aa5b852d74ccde97b9000 2023-07-14T19:22:01.992 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 systemd[1]: Started Ceph mgr.z for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:22:01.992 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:01.845+0000 7f457e99e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:22:01.992 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:01.929+0000 7f457e99e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:22:02.009 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-14T19:22:02.009 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-14T19:22:02.009 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 381107 Links: 1 2023-07-14T19:22:02.009 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:22:02.009 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-07-14T19:22:02.010 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-07-14 19:22:01.101459770 +0000 2023-07-14T19:22:02.010 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-07-14 19:08:12.416836405 +0000 2023-07-14T19:22:02.010 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-07-14 19:08:12.416836405 +0000 2023-07-14T19:22:02.010 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-07-14T19:22:02.011 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-14T19:22:02.075 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-07-14T19:22:02.075 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-07-14T19:22:02.076 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000236082 s, 2.2 MB/s 2023-07-14T19:22:02.077 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-14T19:22:02.134 INFO:tasks.cephadm:Deploying osd.0 on smithi082 with /dev/vg_nvme/lv_4... 2023-07-14T19:22:02.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T19:22:02.292 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.022+0000 7f457e99e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:22:02.292 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.120+0000 7f457e99e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:22:02.696 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:22:02.911 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.290+0000 7f457e99e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:22:02.911 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.596+0000 7f457e99e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:22:02.912 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.684+0000 7f457e99e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:22:02.912 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.814+0000 7f457e99e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:22:02.912 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:02.913 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:02.913 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:02.913 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:02.914 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T19:22:02.914 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:02.914 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:02.915 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-mon[129184]: Deploying daemon mgr.y on smithi133 2023-07-14T19:22:03.028 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:03.028 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:03.028 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:03.029 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:03.029 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T19:22:03.029 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:03.029 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:03.029 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:02 smithi133 ceph-mon[128894]: Deploying daemon mgr.y on smithi133 2023-07-14T19:22:03.073 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:03.073 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:03.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:03.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:03.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-14T19:22:03.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:03.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:03.074 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:02 smithi082 ceph-mon[127461]: Deploying daemon mgr.y on smithi133 2023-07-14T19:22:03.241 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:02.909+0000 7f457e99e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:22:03.568 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:03 smithi133 systemd[1]: Starting Ceph mgr.y for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:22:03.742 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:03.330+0000 7f457e99e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:22:03.917 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:03 smithi133 podman[130327]: 2023-07-14T19:22:03.917 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:03 smithi133 bash[130327]: 1f6b6186935a4b9c02bc0305550311c6af73f407b220473abc5e41a071402293 2023-07-14T19:22:03.917 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:03 smithi133 ceph-mgr[130367]: -- 172.21.15.133:0/79172567 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b0e46b1e0 con 0x555b0e440c00 2023-07-14T19:22:03.918 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:03 smithi133 systemd[1]: Started Ceph mgr.y for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:22:04.183 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:03.915+0000 7f94ce053200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:22:04.184 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.000+0000 7f94ce053200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:22:04.184 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.101+0000 7f94ce053200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:22:04.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-mon[128894]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:04.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:04.206 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T19:22:04.207 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T19:22:04.208 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-07-14T19:22:04.208 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-07-14T19:22:04.208 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012718 s, 824 MB/s 2023-07-14T19:22:04.208 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-07-14T19:22:04.242 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:03.906+0000 7f457e99e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:22:04.242 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:04.090+0000 7f457e99e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:22:04.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-mon[129184]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:04.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:04.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:04 smithi082 ceph-mon[127461]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:04.344 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.345 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.345 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.345 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:04.345 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:04.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2023-07-14T19:22:04.683 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.199+0000 7f94ce053200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:22:04.684 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.377+0000 7f94ce053200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:22:05.024 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.698+0000 7f94ce053200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:22:05.024 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.790+0000 7f94ce053200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:22:05.025 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:04.923+0000 7f94ce053200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:22:05.104 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:22:05.167 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:04.891+0000 7f457e99e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:22:05.167 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:04.986+0000 7f457e99e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:22:05.460 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:05.165+0000 7f457e99e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:22:05.742 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:05.458+0000 7f457e99e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:22:05.742 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:05.692+0000 7f457e99e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:22:05.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:05.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:05.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:05.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:05.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:05.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:05.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.936 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:05.022+0000 7f94ce053200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:22:05.936 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:05.468+0000 7f94ce053200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:22:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:05.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:05.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:05.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:06.101 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:05.842+0000 7f457e99e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:22:06.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 -- 172.21.15.82:0/3257983891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00ffdb0 msgr2=0x7fd6d0100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:06.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3257983891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d0100200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009990 tx=0x7fd6c0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:22:06.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 -- 172.21.15.82:0/3257983891 shutdown_connections 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3257983891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d0100740 0x7fd6d0107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3257983891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d00febb0 0x7fd6d00fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3257983891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d0100200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 -- 172.21.15.82:0/3257983891 >> 172.21.15.82:0/3257983891 conn(0x7fd6d00fa1e0 msgr2=0x7fd6d00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 -- 172.21.15.82:0/3257983891 shutdown_connections 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.335+0000 7fd6d7183700 1 -- 172.21.15.82:0/3257983891 wait complete. 2023-07-14T19:22:06.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 Processor -- start 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 -- start start 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 0x7fd6d010a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d010aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d0100740 0x7fd6d0111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01121f0 con 0x7fd6d00febb0 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0112360 con 0x7fd6d0100740 2023-07-14T19:22:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d7183700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0112520 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d4f1f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 0x7fd6d010a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6cffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d010aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6cffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d010aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:49110/0 (socket says 172.21.15.82:49110) 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d4f1f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 0x7fd6d010a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39844/0 (socket says 172.21.15.82:39844) 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6cffff700 1 -- 172.21.15.82:0/3431719999 learned_addr learned my addr 172.21.15.82:0/3431719999 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.336+0000 7fd6d5720700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d0100740 0x7fd6d0111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cffff700 1 -- 172.21.15.82:0/3431719999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d0100740 msgr2=0x7fd6d0111c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cffff700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d0100740 0x7fd6d0111c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cffff700 1 -- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 msgr2=0x7fd6d010a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cffff700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 0x7fd6d010a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cffff700 1 -- 172.21.15.82:0/3431719999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00095f0 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6d4f1f700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 0x7fd6d010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cffff700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d010aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00059d0 tx=0x7fd6c0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c001d070 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c000fb90 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01127a0 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00173f0 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.337+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d0112c40 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.338+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd6c0017550 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.338+0000 7fd6cdffb700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd6bc0389e0 0x7fd6bc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.338+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd6c004d640 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.338+0000 7fd6d4f1f700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd6bc0389e0 0x7fd6bc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.338+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d006c890 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.339+0000 7fd6d4f1f700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd6bc0389e0 0x7fd6bc03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00ffc10 tx=0x7fd6b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:22:06.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.344+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd6c0027080 con 0x7fd6d00ffdb0 2023-07-14T19:22:06.434 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:06.045+0000 7f94ce053200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:22:06.434 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:06.215+0000 7f94ce053200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:22:06.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:06.100+0000 7f457e99e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:22:06.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:06.549+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d00717d0 con 0x7fd6bc0389e0 2023-07-14T19:22:06.952 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T19:22:06.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:06.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:06.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:06.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: Reconfiguring daemon mgr.x on smithi082 2023-07-14T19:22:06.953 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:06.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:06.954 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:06.979 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T19:22:06.980 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:06.980 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:06.980 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:06.980 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: Reconfiguring daemon mgr.x on smithi082 2023-07-14T19:22:06.980 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:06.980 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:06.981 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:06.994 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:06.799+0000 7f457e99e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:22:06.994 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:06.928+0000 7f457e99e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:22:06.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: Reconfiguring mgr.x (unknown last config time)... 2023-07-14T19:22:06.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-14T19:22:06.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-14T19:22:06.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:06.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: Reconfiguring daemon mgr.x on smithi082 2023-07-14T19:22:06.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:06.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:06.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:07.240 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:06.978+0000 7f94ce053200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:22:07.240 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:07.069+0000 7f94ce053200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:22:07.447 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:07.119+0000 7f457e99e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:22:07.447 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:07.231+0000 7f457e99e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:22:07.516 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:07.238+0000 7f94ce053200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:22:07.741 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:07.446+0000 7f457e99e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:22:07.893 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:07.514+0000 7f94ce053200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:22:07.893 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:07.746+0000 7f94ce053200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:22:08.152 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:07.892+0000 7f94ce053200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:22:08.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:08.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:08.152 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.153 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.153 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:08.153 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:08.153 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:08.153 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.280 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:08.280 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:08.280 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.280 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.281 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:08.281 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:08.281 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:08.281 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:08.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:08.301 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:08.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:08.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:08.302 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:08.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:08.150+0000 7f94ce053200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:22:08.741 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:22:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:22:08.300+0000 7f457e99e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:22:09.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:09.028+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7fd6c000fd00 con 0x7fd6d00ffdb0 2023-07-14T19:22:09.172 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:08.847+0000 7f94ce053200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:22:09.172 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:08.977+0000 7f94ce053200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:22:09.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: Standby manager daemon z started 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3926921878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed"}]: dispatch 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3926921878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed"}]': finished 2023-07-14T19:22:09.173 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T19:22:09.174 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:09.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:09.169+0000 7f94ce053200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:22:09.434 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:09.282+0000 7f94ce053200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:22:09.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: Standby manager daemon z started 2023-07-14T19:22:09.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T19:22:09.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:22:09.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T19:22:09.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:22:09.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3926921878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed"}]: dispatch 2023-07-14T19:22:09.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3926921878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed"}]': finished 2023-07-14T19:22:09.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T19:22:09.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:09.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: Standby manager daemon z started 2023-07-14T19:22:09.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T19:22:09.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:22:09.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T19:22:09.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/1403248796' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:22:09.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3926921878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed"}]: dispatch 2023-07-14T19:22:09.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3926921878' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed"}]': finished 2023-07-14T19:22:09.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: osdmap e5: 1 total, 0 up, 1 in 2023-07-14T19:22:09.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:09.933 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:09.499+0000 7f94ce053200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:22:10.368 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:10 smithi133 ceph-mon[128894]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:10.369 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:10 smithi133 ceph-mon[128894]: mgrmap e14: x(active, since 64s), standbys: z 2023-07-14T19:22:10.369 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T19:22:10.369 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:10 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/384649468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:10.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:10 smithi082 ceph-mon[127461]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:10.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:10 smithi082 ceph-mon[127461]: mgrmap e14: x(active, since 64s), standbys: z 2023-07-14T19:22:10.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T19:22:10.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:10 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/384649468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:10.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:10 smithi161 ceph-mon[129184]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:10.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:10 smithi161 ceph-mon[129184]: mgrmap e14: x(active, since 64s), standbys: z 2023-07-14T19:22:10.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T19:22:10.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:10 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/384649468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:10.685 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:22:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:22:10.366+0000 7f94ce053200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:22:11.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:11.038+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mon.1 v2:172.21.15.161:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fd6c0020370 con 0x7fd6d00ffdb0 2023-07-14T19:22:11.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:11 smithi133 ceph-mon[128894]: Standby manager daemon y started 2023-07-14T19:22:11.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:11 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T19:22:11.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:11 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:22:11.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:11 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T19:22:11.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:11 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:22:11.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:11 smithi082 ceph-mon[127461]: Standby manager daemon y started 2023-07-14T19:22:11.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:11 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T19:22:11.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:11 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:22:11.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:11 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T19:22:11.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:11 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:22:11.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:11 smithi161 ceph-mon[129184]: Standby manager daemon y started 2023-07-14T19:22:11.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:11 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T19:22:11.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:11 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:22:11.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:11 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T19:22:11.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:11 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/2665966812' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:22:12.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:12 smithi133 ceph-mon[128894]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:12.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:12 smithi133 ceph-mon[128894]: mgrmap e15: x(active, since 66s), standbys: z, y 2023-07-14T19:22:12.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T19:22:12.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:12 smithi082 ceph-mon[127461]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:12.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:12 smithi082 ceph-mon[127461]: mgrmap e15: x(active, since 66s), standbys: z, y 2023-07-14T19:22:12.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T19:22:12.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:12 smithi161 ceph-mon[129184]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:12.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:12 smithi161 ceph-mon[129184]: mgrmap e15: x(active, since 66s), standbys: z, y 2023-07-14T19:22:12.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T19:22:14.292 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:14 smithi082 ceph-mon[127461]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:14.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:14 smithi133 ceph-mon[128894]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:14.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:14 smithi161 ceph-mon[129184]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:16.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:16 smithi082 ceph-mon[127461]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:16.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T19:22:16.133 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:16.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:16 smithi133 ceph-mon[128894]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:16.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:16 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T19:22:16.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:16 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:16 smithi161 ceph-mon[129184]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:16 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-14T19:22:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:16 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:17.322 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:17 smithi082 ceph-mon[127461]: Deploying daemon osd.0 on smithi082 2023-07-14T19:22:17.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:17 smithi133 ceph-mon[128894]: Deploying daemon osd.0 on smithi082 2023-07-14T19:22:17.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:17 smithi161 ceph-mon[129184]: Deploying daemon osd.0 on smithi082 2023-07-14T19:22:18.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:18 smithi133 ceph-mon[128894]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:18.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:18 smithi082 ceph-mon[127461]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:18.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:18 smithi161 ceph-mon[129184]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:20.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:20 smithi133 ceph-mon[128894]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:20.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:20 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:20.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:20 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:20.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:20 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:20.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:20 smithi082 ceph-mon[127461]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:20.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:20 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:20.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:20 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:20.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:20 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:20.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:20 smithi161 ceph-mon[129184]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:20.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:20 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:20.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:20 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:20.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:20 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:22 smithi082 ceph-mon[127461]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:22.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:22 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:22 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:22 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:22.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:22 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:22.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:22 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:22 smithi133 ceph-mon[128894]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:22.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:22 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:22 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:22 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:22.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:22 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:22.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:22 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:22 smithi161 ceph-mon[129184]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:22.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:22 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:22 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:22 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:22.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:22 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:22.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:22 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:22.509 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 0 on host 'smithi082' 2023-07-14T19:22:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.507+0000 7fd6cdffb700 1 -- 172.21.15.82:0/3431719999 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd6d00717d0 con 0x7fd6bc0389e0 2023-07-14T19:22:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd6bc0389e0 msgr2=0x7fd6bc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd6bc0389e0 0x7fd6bc03ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00ffc10 tx=0x7fd6b8009260 comp rx=0 tx=0).stop 2023-07-14T19:22:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 msgr2=0x7fd6d010aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d010aba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00059d0 tx=0x7fd6c0004730 comp rx=0 tx=0).stop 2023-07-14T19:22:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 shutdown_connections 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6d0100740 0x7fd6d0111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd6bc0389e0 0x7fd6bc03ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6d00febb0 0x7fd6d010a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 --2- 172.21.15.82:0/3431719999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6d00ffdb0 0x7fd6d010aba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 >> 172.21.15.82:0/3431719999 conn(0x7fd6d00fa1e0 msgr2=0x7fd6d00fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 shutdown_connections 2023-07-14T19:22:22.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:22:22.509+0000 7fd6d7183700 1 -- 172.21.15.82:0/3431719999 wait complete. 2023-07-14T19:22:22.987 DEBUG:teuthology.orchestra.run.smithi082:osd.0> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.0.service 2023-07-14T19:22:22.991 INFO:tasks.cephadm:Deploying osd.1 on smithi133 with /dev/vg_nvme/lv_4... 2023-07-14T19:22:22.991 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T19:22:23.020 INFO:journalctl@ceph.osd.0.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:22:23.558 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:22:23.714 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:22:23 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0[140455]: 2023-07-14T19:22:23.651+0000 7fa2bc172640 -1 osd.0 0 log_to_monitors true 2023-07-14T19:22:23.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:23.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:23.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:23.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:23 smithi082 ceph-mon[127461]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:23.756 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:23.756 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:23.756 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:23.756 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:23 smithi133 ceph-mon[128894]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:23.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:23 smithi161 ceph-mon[129184]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:24.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:24 smithi082 ceph-mon[127461]: from='osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T19:22:24.803 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:24 smithi082 ceph-mon[127461]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T19:22:24.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:24 smithi133 ceph-mon[128894]: from='osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T19:22:24.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:24 smithi133 ceph-mon[128894]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T19:22:24.936 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T19:22:24.936 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T19:22:24.936 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T19:22:24.936 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T19:22:24.936 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111881 s, 937 MB/s 2023-07-14T19:22:24.936 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T19:22:24.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:24 smithi161 ceph-mon[129184]: from='osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T19:22:24.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:24 smithi161 ceph-mon[129184]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-14T19:22:25.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2023-07-14T19:22:25.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T19:22:25.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T19:22:25.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:25.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:25.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:25.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.859 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:22:25.964 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0[140455]: 2023-07-14T19:22:25.519+0000 7fa2b17ea700 -1 osd.0 0 waiting for initial osdmap 2023-07-14T19:22:25.965 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0[140455]: 2023-07-14T19:22:25.522+0000 7fa2aaddb700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:22:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T19:22:25.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T19:22:25.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:25.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:22:25.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:22:25.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:25.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:22:25.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:25.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:25.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-14T19:22:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: osdmap e6: 1 total, 0 up, 1 in 2023-07-14T19:22:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:22:25.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:22:25.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:25.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:26.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:26 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi082 2023-07-14T19:22:26.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:26 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi082 to 17168M 2023-07-14T19:22:26.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:26 smithi133 ceph-mon[128894]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:22:26.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:26 smithi133 ceph-mon[128894]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T19:22:26.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:26 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:26.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:26 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.938+0000 7fd2ae19c700 1 -- 172.21.15.133:0/476871340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 msgr2=0x7fd2a8107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.938+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/476871340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8107070 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000b0a0 tx=0x7fd2a000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:22:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 -- 172.21.15.133:0/476871340 shutdown_connections 2023-07-14T19:22:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/476871340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8107070 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/476871340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 0x7fd2a8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/476871340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 0x7fd2a80ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 -- 172.21.15.133:0/476871340 >> 172.21.15.133:0/476871340 conn(0x7fd2a80f9990 msgr2=0x7fd2a80fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:22:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 -- 172.21.15.133:0/476871340 shutdown_connections 2023-07-14T19:22:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 -- 172.21.15.133:0/476871340 wait complete. 2023-07-14T19:22:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.939+0000 7fd2ae19c700 1 Processor -- start 2023-07-14T19:22:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 -- start start 2023-07-14T19:22:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 0x7fd2a8194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 0x7fd2a819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a819c500 con 0x7fd2a8100240 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a819c640 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a819c7d0 con 0x7fd2a80fee10 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8194eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38832/0 (socket says 172.21.15.133:38832) 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 -- 172.21.15.133:0/4236223537 learned_addr learned my addr 172.21.15.133:0/4236223537 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a77fe700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 0x7fd2a8194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a7fff700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 0x7fd2a819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 -- 172.21.15.133:0/4236223537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 msgr2=0x7fd2a8194970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 0x7fd2a8194970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 -- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 msgr2=0x7fd2a819bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 0x7fd2a819bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 -- 172.21.15.133:0/4236223537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009d00 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a77fe700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 0x7fd2a8194970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a7fff700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 0x7fd2a819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a6ffd700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8194eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009970 tx=0x7fd29c009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c012070 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd29c00d260 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.940+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80fdd60 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.941+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c017a60 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.941+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a80fe2b0 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.942+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a810a380 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.943+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fd29c017bc0 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.943+0000 7fd2a4ff9700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2900a1140 0x7fd2900a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.943+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd29c0cc1b0 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.943+0000 7fd2a77fe700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2900a1140 0x7fd2900a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.944+0000 7fd2a77fe700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2900a1140 0x7fd2900a35f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd298005fd0 tx=0x7fd298005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:22:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:26.947+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd29c097370 con 0x7fd2a80ff7c0 2023-07-14T19:22:26.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:26 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi082 2023-07-14T19:22:26.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:26 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi082 to 17168M 2023-07-14T19:22:26.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:26 smithi082 ceph-mon[127461]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:22:26.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:26 smithi082 ceph-mon[127461]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T19:22:26.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:26 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:26.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:26 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:26.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:26 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi082 2023-07-14T19:22:26.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:26 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi082 to 17168M 2023-07-14T19:22:26.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:26 smithi161 ceph-mon[129184]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:22:26.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:26 smithi161 ceph-mon[129184]: osdmap e7: 1 total, 0 up, 1 in 2023-07-14T19:22:26.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:26 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:26.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:26 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:27.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:27.141+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2a8060a80 con 0x7fd2900a1140 2023-07-14T19:22:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:22:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:22:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] boot 2023-07-14T19:22:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T19:22:27.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:27.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:27.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:27.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:27.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:27.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:22:27.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:22:27.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] boot 2023-07-14T19:22:27.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T19:22:27.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:27.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:27.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:27.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:27.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:27.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:22:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:22:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: osd.0 [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] boot 2023-07-14T19:22:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: osdmap e8: 1 total, 1 up, 1 in 2023-07-14T19:22:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:22:27.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-14T19:22:27.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:27.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:27.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:28.774 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:28 smithi133 ceph-mon[128894]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:28.775 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:28 smithi133 ceph-mon[128894]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T19:22:28.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:28 smithi082 ceph-mon[127461]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:28.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:28 smithi082 ceph-mon[127461]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T19:22:28.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:28 smithi161 ceph-mon[129184]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:28.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:28 smithi161 ceph-mon[129184]: osdmap e9: 1 total, 1 up, 1 in 2023-07-14T19:22:29.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:29 smithi133 ceph-mon[128894]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:29.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:29 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/500698702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]: dispatch 2023-07-14T19:22:29.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:29 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]: dispatch 2023-07-14T19:22:29.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:29 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]': finished 2023-07-14T19:22:29.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:29 smithi133 ceph-mon[128894]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T19:22:29.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:29.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:29 smithi082 ceph-mon[127461]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:29.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:29 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/500698702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]: dispatch 2023-07-14T19:22:29.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:29 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]: dispatch 2023-07-14T19:22:29.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:29 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]': finished 2023-07-14T19:22:29.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:29 smithi082 ceph-mon[127461]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T19:22:29.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:29.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:29 smithi161 ceph-mon[129184]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:29 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/500698702' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]: dispatch 2023-07-14T19:22:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:29 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]: dispatch 2023-07-14T19:22:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:29 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "087df2cd-4b6e-4ca5-a4ac-13083d804ef4"}]': finished 2023-07-14T19:22:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:29 smithi161 ceph-mon[129184]: osdmap e10: 2 total, 1 up, 2 in 2023-07-14T19:22:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:30.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:30 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/3010949628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:30.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:30 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/3010949628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:30.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:30 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/3010949628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:31.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:31 smithi133 ceph-mon[128894]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:31.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:31 smithi082 ceph-mon[127461]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:31.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:31 smithi161 ceph-mon[129184]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:34.293 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:34 smithi133 ceph-mon[128894]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:34.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:34 smithi082 ceph-mon[127461]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:34.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:34 smithi161 ceph-mon[129184]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:36.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:36 smithi133 ceph-mon[128894]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:36 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T19:22:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:36 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:36.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:36 smithi082 ceph-mon[127461]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:36.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:36 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T19:22:36.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:36 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:36.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:36 smithi161 ceph-mon[129184]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:36.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:36 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-14T19:22:36.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:36 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:37.302 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:37 smithi133 ceph-mon[128894]: Deploying daemon osd.1 on smithi133 2023-07-14T19:22:37.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:37 smithi082 ceph-mon[127461]: Deploying daemon osd.1 on smithi133 2023-07-14T19:22:37.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:37 smithi161 ceph-mon[129184]: Deploying daemon osd.1 on smithi133 2023-07-14T19:22:38.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:38 smithi133 ceph-mon[128894]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:38.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:38 smithi082 ceph-mon[127461]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:38.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:38 smithi161 ceph-mon[129184]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:40.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:40 smithi133 ceph-mon[128894]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:40.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:40 smithi082 ceph-mon[127461]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:40.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:40 smithi161 ceph-mon[129184]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:41 smithi133 ceph-mon[128894]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:41.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:41.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:41.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:41.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:41 smithi082 ceph-mon[127461]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:41.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:41.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:41.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:41.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:41 smithi161 ceph-mon[129184]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:42.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.702+0000 7fd2a4ff9700 1 -- 172.21.15.133:0/4236223537 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd2a8060a80 con 0x7fd2900a1140 2023-07-14T19:22:42.704 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 1 on host 'smithi133' 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2900a1140 msgr2=0x7fd2900a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2900a1140 0x7fd2900a35f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd298005fd0 tx=0x7fd298005dc0 comp rx=0 tx=0).stop 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 msgr2=0x7fd2a8194eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8194eb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009970 tx=0x7fd29c009d30 comp rx=0 tx=0).stop 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 shutdown_connections 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2900a1140 0x7fd2900a35f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2a8100240 0x7fd2a819bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2a80fee10 0x7fd2a8194970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 --2- 172.21.15.133:0/4236223537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2a80ff7c0 0x7fd2a8194eb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.704+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 >> 172.21.15.133:0/4236223537 conn(0x7fd2a80f9990 msgr2=0x7fd2a80fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:22:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.705+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 shutdown_connections 2023-07-14T19:22:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:22:42.705+0000 7fd2ae19c700 1 -- 172.21.15.133:0/4236223537 wait complete. 2023-07-14T19:22:43.055 DEBUG:teuthology.orchestra.run.smithi133:osd.1> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.1.service 2023-07-14T19:22:43.057 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2023-07-14T19:22:43.058 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-14T19:22:43.088 INFO:journalctl@ceph.osd.1.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:22:43.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:43.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:43.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:43.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:43.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:43.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:43.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:43.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:22:43.621 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:22:44.183 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:22:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1[133886]: 2023-07-14T19:22:44.086+0000 7fe30429a640 -1 osd.1 0 log_to_monitors true 2023-07-14T19:22:44.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:44 smithi133 ceph-mon[128894]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:44.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:44 smithi133 ceph-mon[128894]: from='osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T19:22:44.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:44 smithi133 ceph-mon[128894]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T19:22:44.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:44 smithi082 ceph-mon[127461]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:44.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:44 smithi082 ceph-mon[127461]: from='osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T19:22:44.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:44 smithi082 ceph-mon[127461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T19:22:44.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:44 smithi161 ceph-mon[129184]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:44.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:44 smithi161 ceph-mon[129184]: from='osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T19:22:44.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:44 smithi161 ceph-mon[129184]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-14T19:22:45.130 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-14T19:22:45.130 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-14T19:22:45.130 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T19:22:45.131 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T19:22:45.131 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113787 s, 922 MB/s 2023-07-14T19:22:45.131 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T19:22:45.184 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1[133886]: 2023-07-14T19:22:45.120+0000 7fe2f9912700 -1 osd.1 0 waiting for initial osdmap 2023-07-14T19:22:45.184 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1[133886]: 2023-07-14T19:22:45.122+0000 7fe2f1f01700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:22:45.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-mon[128894]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T19:22:45.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-mon[128894]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T19:22:45.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:45.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-mon[128894]: from='osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:22:45.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-mon[128894]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:22:45.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:45 smithi133 ceph-mon[128894]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:45.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:45 smithi082 ceph-mon[127461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T19:22:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:45 smithi082 ceph-mon[127461]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T19:22:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:45 smithi082 ceph-mon[127461]: from='osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:22:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:45 smithi082 ceph-mon[127461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:22:45.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:45 smithi082 ceph-mon[127461]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:45.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:45 smithi161 ceph-mon[129184]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-14T19:22:45.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:45 smithi161 ceph-mon[129184]: osdmap e11: 2 total, 1 up, 2 in 2023-07-14T19:22:45.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:45.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:45 smithi161 ceph-mon[129184]: from='osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:22:45.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:45 smithi161 ceph-mon[129184]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:22:45.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:45 smithi161 ceph-mon[129184]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:45.572 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2023-07-14T19:22:46.120 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:22:46.392 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:22:46.392 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi133 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:22:46.393 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi133 to 17168M 2023-07-14T19:22:46.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:46.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:46.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:46 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:22:46.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T19:22:46.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi133 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi133 to 17168M 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:46.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:46.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:46 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:22:46.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: osdmap e12: 2 total, 1 up, 2 in 2023-07-14T19:22:46.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:46.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:46.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi133 2023-07-14T19:22:46.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi133 to 17168M 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:22:46.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:46 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:22:47.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 -- 172.21.15.161:0/695156189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 msgr2=0x7f419c1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:47.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 --2- 172.21.15.161:0/695156189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c1005e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f418c009990 tx=0x7f418c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:22:47.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 -- 172.21.15.161:0/695156189 shutdown_connections 2023-07-14T19:22:47.430 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 --2- 172.21.15.161:0/695156189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 0x7f419c107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 --2- 172.21.15.161:0/695156189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c1005e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 --2- 172.21.15.161:0/695156189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f419c0fee30 0x7f419c0ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 -- 172.21.15.161:0/695156189 >> 172.21.15.161:0/695156189 conn(0x7f419c0f9990 msgr2=0x7f419c0fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 -- 172.21.15.161:0/695156189 shutdown_connections 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.426+0000 7f41a29c0700 1 -- 172.21.15.161:0/695156189 wait complete. 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 Processor -- start 2023-07-14T19:22:47.431 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 -- start start 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f419c0fee30 0x7f419c0fe880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 0x7f419c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c19d520 con 0x7f419c0ff7e0 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c19d690 con 0x7f419c0fee30 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a29c0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c19d850 con 0x7f419c1001d0 2023-07-14T19:22:47.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60134/0 (socket says 172.21.15.161:60134) 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 -- 172.21.15.161:0/1404466695 learned_addr learned my addr 172.21.15.161:0/1404466695 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f419bfff700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f419c0fee30 0x7f419c0fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f419b7fe700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 0x7f419c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 -- 172.21.15.161:0/1404466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f419c0fee30 msgr2=0x7f419c0fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f419c0fee30 0x7f419c0fe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:47.433 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 -- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 msgr2=0x7f419c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 0x7f419c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.427+0000 7f41a0f5d700 1 -- 172.21.15.161:0/1404466695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0095f0 con 0x7f419c1001d0 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f419b7fe700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 0x7f419c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f41a0f5d700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c19cd80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4194007ee0 tx=0x7f419400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194017070 con 0x7f419c1001d0 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f419400f040 con 0x7f419c1001d0 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c19db30 con 0x7f419c1001d0 2023-07-14T19:22:47.434 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194005740 con 0x7f419c1001d0 2023-07-14T19:22:47.435 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.428+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c19dfe0 con 0x7f419c1001d0 2023-07-14T19:22:47.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:22:47.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:22:47.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:47.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] boot 2023-07-14T19:22:47.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T19:22:47.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:47.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:47 smithi133 ceph-mon[128894]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:47.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.429+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c106210 con 0x7f419c1001d0 2023-07-14T19:22:47.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.430+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f41940058a0 con 0x7f419c1001d0 2023-07-14T19:22:47.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.430+0000 7f41997fa700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41840a10c0 0x7f41840a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:22:47.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.430+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f41940ca940 con 0x7f419c1001d0 2023-07-14T19:22:47.437 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.431+0000 7f419bfff700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41840a10c0 0x7f41840a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:22:47.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.431+0000 7f419bfff700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41840a10c0 0x7f41840a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f418c00b5c0 tx=0x7f418c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:22:47.438 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.435+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f41940958e0 con 0x7f419c1001d0 2023-07-14T19:22:47.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:22:47.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:22:47.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:47.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] boot 2023-07-14T19:22:47.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T19:22:47.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:47.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:47 smithi082 ceph-mon[127461]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:47.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:22:47.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:22:47.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:47.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: osd.1 [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] boot 2023-07-14T19:22:47.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: osdmap e13: 2 total, 2 up, 2 in 2023-07-14T19:22:47.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:22:47.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:47 smithi161 ceph-mon[129184]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-14T19:22:47.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:22:47.626+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f419c060a80 con 0x7f41840a10c0 2023-07-14T19:22:48.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:48 smithi133 ceph-mon[128894]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T19:22:48.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:48 smithi133 ceph-mon[128894]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:48.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:48.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:48.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:48.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:48 smithi161 ceph-mon[129184]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T19:22:48.439 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:48 smithi161 ceph-mon[129184]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:48.440 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:48.440 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:48.440 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:48.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:48 smithi082 ceph-mon[127461]: osdmap e14: 2 total, 2 up, 2 in 2023-07-14T19:22:48.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:48 smithi082 ceph-mon[127461]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:22:48.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:22:48.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:22:48.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:49.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:49 smithi133 ceph-mon[128894]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:49.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:49 smithi082 ceph-mon[127461]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:49.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:49 smithi161 ceph-mon[129184]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:50.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:50 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/3480237059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]: dispatch 2023-07-14T19:22:50.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:50 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]: dispatch 2023-07-14T19:22:50.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:50 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]': finished 2023-07-14T19:22:50.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:50 smithi133 ceph-mon[128894]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T19:22:50.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:22:50.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:50 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/3480237059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]: dispatch 2023-07-14T19:22:50.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:50 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]: dispatch 2023-07-14T19:22:50.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:50 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]': finished 2023-07-14T19:22:50.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:50 smithi082 ceph-mon[127461]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T19:22:50.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:22:50.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:50 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/3480237059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]: dispatch 2023-07-14T19:22:50.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:50 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]: dispatch 2023-07-14T19:22:50.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:50 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2"}]': finished 2023-07-14T19:22:50.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:50 smithi161 ceph-mon[129184]: osdmap e15: 3 total, 2 up, 3 in 2023-07-14T19:22:50.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:22:51.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:51 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/1419720941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:51.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:51 smithi133 ceph-mon[128894]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:51.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:51 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/1419720941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:51.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:51 smithi082 ceph-mon[127461]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:51.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:51 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/1419720941' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:22:51.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:51 smithi161 ceph-mon[129184]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:54.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:54 smithi133 ceph-mon[128894]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:54.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:54 smithi082 ceph-mon[127461]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:54.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:54 smithi161 ceph-mon[129184]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:56.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:56 smithi161 ceph-mon[129184]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:56.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:56 smithi133 ceph-mon[128894]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:56.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:56 smithi082 ceph-mon[127461]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:57.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T19:22:57.329 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:57.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T19:22:57.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:57.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-14T19:22:57.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:22:58.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:58 smithi161 ceph-mon[129184]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:58.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:22:58 smithi161 ceph-mon[129184]: Deploying daemon osd.2 on smithi161 2023-07-14T19:22:58.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:58 smithi133 ceph-mon[128894]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:58.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:22:58 smithi133 ceph-mon[128894]: Deploying daemon osd.2 on smithi161 2023-07-14T19:22:58.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:58 smithi082 ceph-mon[127461]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:22:58.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:22:58 smithi082 ceph-mon[127461]: Deploying daemon osd.2 on smithi161 2023-07-14T19:23:00.356 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:00 smithi161 ceph-mon[129184]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:00.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:00 smithi133 ceph-mon[128894]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:00.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:00 smithi082 ceph-mon[127461]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:02.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:02 smithi161 ceph-mon[129184]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:02.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:02.391 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:02.392 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:02.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:02 smithi133 ceph-mon[128894]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:02.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:02.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:02.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:02.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:02 smithi082 ceph-mon[127461]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:02.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:02.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:02.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.237 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.237+0000 7f41997fa700 1 -- 172.21.15.161:0/1404466695 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f419c060a80 con 0x7f41840a10c0 2023-07-14T19:23:04.238 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2023-07-14T19:23:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41840a10c0 msgr2=0x7f41840a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41840a10c0 0x7f41840a3570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f418c00b5c0 tx=0x7f418c005fb0 comp rx=0 tx=0).stop 2023-07-14T19:23:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 msgr2=0x7f419c19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c19cd80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4194007ee0 tx=0x7f419400d3b0 comp rx=0 tx=0).stop 2023-07-14T19:23:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 shutdown_connections 2023-07-14T19:23:04.240 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f419c1001d0 0x7f419c19cd80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f419c0fee30 0x7f419c0fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41840a10c0 0x7f41840a3570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 --2- 172.21.15.161:0/1404466695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f419c0ff7e0 0x7f419c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 >> 172.21.15.161:0/1404466695 conn(0x7f419c0f9990 msgr2=0x7f419c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:23:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 shutdown_connections 2023-07-14T19:23:04.241 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:04.239+0000 7f41a29c0700 1 -- 172.21.15.161:0/1404466695 wait complete. 2023-07-14T19:23:04.413 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:04 smithi161 ceph-mon[129184]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:04.413 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.413 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.414 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:04.414 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:04.414 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:04 smithi133 ceph-mon[128894]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:04.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:04.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:04.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:04 smithi082 ceph-mon[127461]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:04.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:04.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:04.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:04.711 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.2.service 2023-07-14T19:23:04.713 INFO:tasks.cephadm:Deploying osd.3 on smithi082 with /dev/vg_nvme/lv_3... 2023-07-14T19:23:04.713 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T19:23:04.746 INFO:journalctl@ceph.osd.2.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:23:05.278 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:23:05.492 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2[134957]: 2023-07-14T19:23:05.149+0000 7f3ef1500640 -1 osd.2 0 log_to_monitors true 2023-07-14T19:23:05.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:05.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:05.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:05.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-mon[129184]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:05.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-mon[129184]: from='osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T19:23:05.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:05 smithi161 ceph-mon[129184]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T19:23:05.512 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:05.512 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:05.512 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:05.513 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:05 smithi082 ceph-mon[127461]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:05.513 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:05 smithi082 ceph-mon[127461]: from='osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T19:23:05.513 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:05 smithi082 ceph-mon[127461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T19:23:05.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:05.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:05.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:05.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:05 smithi133 ceph-mon[128894]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:05.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:05 smithi133 ceph-mon[128894]: from='osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T19:23:05.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:05 smithi133 ceph-mon[128894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-14T19:23:06.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-mon[129184]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T19:23:06.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-mon[129184]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T19:23:06.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:06.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-mon[129184]: from='osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:23:06.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-mon[129184]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:23:06.493 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2[134957]: 2023-07-14T19:23:06.246+0000 7f3ee6b78700 -1 osd.2 0 waiting for initial osdmap 2023-07-14T19:23:06.493 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2[134957]: 2023-07-14T19:23:06.250+0000 7f3edf167700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:23:06.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:06 smithi133 ceph-mon[128894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T19:23:06.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:06 smithi133 ceph-mon[128894]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T19:23:06.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:06.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:06 smithi133 ceph-mon[128894]: from='osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:23:06.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:06 smithi133 ceph-mon[128894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:23:06.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:06 smithi082 ceph-mon[127461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-14T19:23:06.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:06 smithi082 ceph-mon[127461]: osdmap e16: 3 total, 2 up, 3 in 2023-07-14T19:23:06.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:06.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:06 smithi082 ceph-mon[127461]: from='osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:23:06.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:06 smithi082 ceph-mon[127461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:23:06.772 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T19:23:06.772 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T19:23:06.773 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-07-14T19:23:06.773 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-07-14T19:23:06.773 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114724 s, 914 MB/s 2023-07-14T19:23:06.773 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-07-14T19:23:07.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2023-07-14T19:23:07.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:23:07.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T19:23:07.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:07.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:07.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:07.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi161 2023-07-14T19:23:07.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:07.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi161 to 17168M 2023-07-14T19:23:07.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:07.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:07.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:23:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T19:23:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi161 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi161 to 17168M 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:07.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:23:07.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: osdmap e17: 3 total, 2 up, 3 in 2023-07-14T19:23:07.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:07.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:07.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-14T19:23:07.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi161 2023-07-14T19:23:07.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi161 to 17168M 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:07.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:07.769 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:23:08.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:08 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:23:08.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:08 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:23:08.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:08 smithi133 ceph-mon[128894]: osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] boot 2023-07-14T19:23:08.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:08 smithi133 ceph-mon[128894]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T19:23:08.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:08 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:08.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:08 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:23:08.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:08 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:23:08.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:08 smithi082 ceph-mon[127461]: osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] boot 2023-07-14T19:23:08.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:08 smithi082 ceph-mon[127461]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T19:23:08.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:08 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:08.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:08 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:23:08.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:08 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:23:08.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:08 smithi161 ceph-mon[129184]: osd.2 [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] boot 2023-07-14T19:23:08.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:08 smithi161 ceph-mon[129184]: osdmap e18: 3 total, 3 up, 3 in 2023-07-14T19:23:08.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:08 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:23:08.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 -- 172.21.15.82:0/2646581405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 msgr2=0x7fa13c105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:08.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 --2- 172.21.15.82:0/2646581405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 0x7fa13c105440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009990 tx=0x7fa12c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 -- 172.21.15.82:0/2646581405 shutdown_connections 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 --2- 172.21.15.82:0/2646581405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa13c105980 0x7fa13c107de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 --2- 172.21.15.82:0/2646581405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 --2- 172.21.15.82:0/2646581405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 0x7fa13c105440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 -- 172.21.15.82:0/2646581405 >> 172.21.15.82:0/2646581405 conn(0x7fa13c0fa1e0 msgr2=0x7fa13c0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 -- 172.21.15.82:0/2646581405 shutdown_connections 2023-07-14T19:23:08.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.978+0000 7fa1439ba700 1 -- 172.21.15.82:0/2646581405 wait complete. 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 Processor -- start 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 -- start start 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 0x7fa13c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa13c105980 0x7fa13c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19d5c0 con 0x7fa13c100740 2023-07-14T19:23:08.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19d700 con 0x7fa13c103060 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa1439ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19d890 con 0x7fa13c105980 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa141756700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa141756700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39914/0 (socket says 172.21.15.82:39914) 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa141756700 1 -- 172.21.15.82:0/1421492909 learned_addr learned my addr 172.21.15.82:0/1421492909 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.979+0000 7fa141f57700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa13c105980 0x7fa13c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa140f55700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 0x7fa13c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa141756700 1 -- 172.21.15.82:0/1421492909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa13c105980 msgr2=0x7fa13c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:08.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa141756700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa13c105980 0x7fa13c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:08.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa141756700 1 -- 172.21.15.82:0/1421492909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 msgr2=0x7fa13c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:08.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa141756700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 0x7fa13c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:08.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa141756700 1 -- 172.21.15.82:0/1421492909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0095f0 con 0x7fa13c100740 2023-07-14T19:23:08.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa141756700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c195a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b560 tx=0x7fa13800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:23:08.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138011830 con 0x7fa13c100740 2023-07-14T19:23:08.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c0fe650 con 0x7fa13c100740 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa138011e50 con 0x7fa13c100740 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800f2e0 con 0x7fa13c100740 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.980+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c0feb50 con 0x7fa13c100740 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.981+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c10b510 con 0x7fa13c100740 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.982+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fa138010580 con 0x7fa13c100740 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.983+0000 7fa1327fc700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa1280a10c0 0x7fa1280a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.983+0000 7fa140f55700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa1280a10c0 0x7fa1280a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:08.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.983+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fa1380cb2b0 con 0x7fa13c100740 2023-07-14T19:23:08.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.983+0000 7fa140f55700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa1280a10c0 0x7fa1280a3570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:23:08.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:08.987+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa138096030 con 0x7fa13c100740 2023-07-14T19:23:09.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:09.179+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa13c102320 con 0x7fa1280a10c0 2023-07-14T19:23:09.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:09 smithi133 ceph-mon[128894]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T19:23:09.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:09 smithi133 ceph-mon[128894]: pgmap v71: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:09.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:23:09.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:09.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:09.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:09.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:09 smithi082 ceph-mon[127461]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T19:23:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:09 smithi082 ceph-mon[127461]: pgmap v71: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:23:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:09.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:09.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:09 smithi161 ceph-mon[129184]: osdmap e19: 3 total, 3 up, 3 in 2023-07-14T19:23:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:09 smithi161 ceph-mon[129184]: pgmap v71: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:23:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:09.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:09.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:10.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:10 smithi082 ceph-mon[127461]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:10.509 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T19:23:10.510 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:10 smithi082 ceph-mon[127461]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T19:23:10.510 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:23:10.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:10 smithi133 ceph-mon[128894]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:10.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T19:23:10.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:10 smithi133 ceph-mon[128894]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T19:23:10.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:23:10.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:10 smithi161 ceph-mon[129184]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:10.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-14T19:23:10.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:10 smithi161 ceph-mon[129184]: osdmap e20: 3 total, 3 up, 3 in 2023-07-14T19:23:10.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:23:10.780 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143438]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T19:23:10.781 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:10.781 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143438]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:10.781 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143438]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:11.214 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143469]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T19:23:11.215 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143469]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:11.215 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143469]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:11.215 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:23:10 smithi082 sudo[143469]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:11.413 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T19:23:11.414 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:11 smithi133 ceph-mon[128894]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T19:23:11.414 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:11 smithi133 ceph-mon[128894]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:11.414 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:11.415 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136206]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T19:23:11.415 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:11.415 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136206]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:11.415 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136206]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:11.682 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T19:23:11.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:11 smithi161 ceph-mon[129184]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T19:23:11.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:11 smithi161 ceph-mon[129184]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:11.683 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:11.683 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136217]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T19:23:11.684 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:11.684 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136217]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:11.684 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:23:11 smithi133 sudo[136217]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:11.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-14T19:23:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:11 smithi082 ceph-mon[127461]: osdmap e21: 3 total, 3 up, 3 in 2023-07-14T19:23:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:11 smithi082 ceph-mon[127461]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:11.935 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137097]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T19:23:11.935 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:11.936 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137097]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:11.936 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137097]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:12.241 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137100]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T19:23:12.242 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:12.242 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137100]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:12.242 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:23:11 smithi161 sudo[137100]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143543]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T19:23:12.458 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:12.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143543]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:12.459 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143543]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:12.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:12.474+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa138096400 con 0x7fa13c100740 2023-07-14T19:23:12.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143546]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T19:23:12.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143546]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:12.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143546]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:12.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 sudo[143546]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:12.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T19:23:12.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:12.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2359796950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aabce84f-f9f4-45cc-b738-7414623bae2c"}]: dispatch 2023-07-14T19:23:12.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2359796950' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aabce84f-f9f4-45cc-b738-7414623bae2c"}]': finished 2023-07-14T19:23:12.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T19:23:12.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:12.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:12.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:12 smithi082 ceph-mon[127461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:12.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T19:23:12.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:12.725 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2359796950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aabce84f-f9f4-45cc-b738-7414623bae2c"}]: dispatch 2023-07-14T19:23:12.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2359796950' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aabce84f-f9f4-45cc-b738-7414623bae2c"}]': finished 2023-07-14T19:23:12.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T19:23:12.726 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:12.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:12.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 ceph-mon[129184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:12.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137103]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T19:23:12.727 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:12.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137103]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:12.728 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137103]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: osdmap e22: 3 total, 3 up, 3 in 2023-07-14T19:23:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2359796950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aabce84f-f9f4-45cc-b738-7414623bae2c"}]: dispatch 2023-07-14T19:23:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2359796950' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aabce84f-f9f4-45cc-b738-7414623bae2c"}]': finished 2023-07-14T19:23:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: osdmap e23: 4 total, 3 up, 4 in 2023-07-14T19:23:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:12.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:12.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:12 smithi133 ceph-mon[128894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:12.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137106]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T19:23:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137106]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137106]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:12 smithi161 sudo[137106]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:13.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136220]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-14T19:23:13.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136220]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:13.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136220]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:13.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136220]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:13.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4282979421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:13.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:13.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T19:23:13.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:13.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:13.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:13 smithi161 ceph-mon[129184]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136223]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-14T19:23:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-14T19:23:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136223]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-14T19:23:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 sudo[136223]: pam_unix(sudo:session): session closed for user root 2023-07-14T19:23:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4282979421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:13 smithi133 ceph-mon[128894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4282979421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:13.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:13.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-14T19:23:13.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:23:13.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:23:13.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:23:13.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:13 smithi082 ceph-mon[127461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-14T19:23:15.900 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:15 smithi082 ceph-mon[127461]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:15.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:15 smithi161 ceph-mon[129184]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:16.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:15 smithi133 ceph-mon[128894]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:17.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:17 smithi082 ceph-mon[127461]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:17.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:17 smithi161 ceph-mon[129184]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:18.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:17 smithi133 ceph-mon[128894]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:18.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:18 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T19:23:18.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:18 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:19.009 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T19:23:19.009 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:18 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:19.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:18 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-14T19:23:19.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:18 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:19.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:19 smithi161 ceph-mon[129184]: Deploying daemon osd.3 on smithi082 2023-07-14T19:23:19.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:19 smithi161 ceph-mon[129184]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-14T19:23:20.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:19 smithi082 ceph-mon[127461]: Deploying daemon osd.3 on smithi082 2023-07-14T19:23:20.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:19 smithi082 ceph-mon[127461]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-14T19:23:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:19 smithi133 ceph-mon[128894]: Deploying daemon osd.3 on smithi082 2023-07-14T19:23:20.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:19 smithi133 ceph-mon[128894]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-07-14T19:23:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:21 smithi082 ceph-mon[127461]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-14T19:23:22.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:21 smithi133 ceph-mon[128894]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-14T19:23:22.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:21 smithi161 ceph-mon[129184]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-07-14T19:23:23.652 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:23.652 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:23.652 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:23.652 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:23 smithi082 ceph-mon[127461]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-14T19:23:23.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:23.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:23.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:23.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:23 smithi133 ceph-mon[128894]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-14T19:23:23.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:23 smithi161 ceph-mon[129184]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-07-14T19:23:25.233 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 3 on host 'smithi082' 2023-07-14T19:23:25.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.232+0000 7fa1327fc700 1 -- 172.21.15.82:0/1421492909 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa13c102320 con 0x7fa1280a10c0 2023-07-14T19:23:25.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa1280a10c0 msgr2=0x7fa1280a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:25.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa1280a10c0 0x7fa1280a3570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c005ab0 comp rx=0 tx=0).stop 2023-07-14T19:23:25.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 msgr2=0x7fa13c195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c195a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa13800b560 tx=0x7fa13800b870 comp rx=0 tx=0).stop 2023-07-14T19:23:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 shutdown_connections 2023-07-14T19:23:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa13c105980 0x7fa13c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa1280a10c0 0x7fa1280a3570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c100740 0x7fa13c195a30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:25.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 --2- 172.21.15.82:0/1421492909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13c103060 0x7fa13c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:25.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 >> 172.21.15.82:0/1421492909 conn(0x7fa13c0fa1e0 msgr2=0x7fa13c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:23:25.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 shutdown_connections 2023-07-14T19:23:25.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:23:25.234+0000 7fa1439ba700 1 -- 172.21.15.82:0/1421492909 wait complete. 2023-07-14T19:23:25.657 DEBUG:teuthology.orchestra.run.smithi082:osd.3> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.3.service 2023-07-14T19:23:25.659 INFO:tasks.cephadm:Deploying osd.4 on smithi133 with /dev/vg_nvme/lv_3... 2023-07-14T19:23:25.659 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T19:23:25.688 INFO:journalctl@ceph.osd.3.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:23:25.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:25 smithi133 ceph-mon[128894]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-14T19:23:25.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:25 smithi082 ceph-mon[127461]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-14T19:23:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:25 smithi161 ceph-mon[129184]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-14T19:23:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:25.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:23:26.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:26 smithi133 ceph-mon[128894]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T19:23:26.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:26 smithi082 ceph-mon[127461]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T19:23:26.965 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:23:26 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3[145767]: 2023-07-14T19:23:26.485+0000 7facc3630640 -1 osd.3 0 log_to_monitors true 2023-07-14T19:23:26.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:26 smithi161 ceph-mon[129184]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-14T19:23:27.575 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T19:23:27.576 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T19:23:27.576 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T19:23:27.577 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T19:23:27.577 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110658 s, 948 MB/s 2023-07-14T19:23:27.577 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T19:23:27.918 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3[145767]: 2023-07-14T19:23:27.699+0000 7facb8ca8700 -1 osd.3 0 waiting for initial osdmap 2023-07-14T19:23:27.918 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3[145767]: 2023-07-14T19:23:27.707+0000 7facb4a9e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:23:27.919 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-mon[127461]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-14T19:23:27.919 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-mon[127461]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T19:23:27.919 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-mon[127461]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T19:23:27.919 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:27.919 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:27 smithi082 ceph-mon[127461]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:23:27.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:27 smithi133 ceph-mon[128894]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-14T19:23:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:27 smithi133 ceph-mon[128894]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T19:23:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:27 smithi133 ceph-mon[128894]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T19:23:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:27.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:27 smithi133 ceph-mon[128894]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:23:27.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2023-07-14T19:23:27.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:27 smithi161 ceph-mon[129184]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-07-14T19:23:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:27 smithi161 ceph-mon[129184]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-14T19:23:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:27 smithi161 ceph-mon[129184]: osdmap e24: 4 total, 3 up, 4 in 2023-07-14T19:23:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:27.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:27 smithi161 ceph-mon[129184]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:23:28.527 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:23:28.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:23:28.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T19:23:28.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi082 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi082 to 8584M 2023-07-14T19:23:28.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] boot 2023-07-14T19:23:28.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T19:23:28.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.971 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:23:28.971 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi082 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:28.972 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi082 to 8584M 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:28.973 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:28.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:28.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] boot 2023-07-14T19:23:28.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T19:23:28.974 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:23:28.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: osdmap e25: 4 total, 3 up, 4 in 2023-07-14T19:23:28.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:28.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi082 2023-07-14T19:23:28.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:28.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:28.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi082 to 8584M 2023-07-14T19:23:28.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:28.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:28.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:28.997 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:28.997 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:28.997 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:28.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: osd.3 [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] boot 2023-07-14T19:23:28.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: osdmap e26: 4 total, 4 up, 4 in 2023-07-14T19:23:28.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:23:29.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 -- 172.21.15.133:0/1958313507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc00febb0 msgr2=0x7efdc00fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 --2- 172.21.15.133:0/1958313507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc00febb0 0x7efdc00fefc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009990 tx=0x7efdb4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:23:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 -- 172.21.15.133:0/1958313507 shutdown_connections 2023-07-14T19:23:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 --2- 172.21.15.133:0/1958313507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc0100740 0x7efdc0107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 --2- 172.21.15.133:0/1958313507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc00febb0 0x7efdc00fefc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 --2- 172.21.15.133:0/1958313507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdc00ffdb0 0x7efdc0100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 -- 172.21.15.133:0/1958313507 >> 172.21.15.133:0/1958313507 conn(0x7efdc00fa1e0 msgr2=0x7efdc00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:23:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 -- 172.21.15.133:0/1958313507 shutdown_connections 2023-07-14T19:23:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.599+0000 7efdc7244700 1 -- 172.21.15.133:0/1958313507 wait complete. 2023-07-14T19:23:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 Processor -- start 2023-07-14T19:23:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 -- start start 2023-07-14T19:23:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc00febb0 0x7efdc0195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdc00ffdb0 0x7efdc0195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 0x7efdc019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc019d5d0 con 0x7efdc00febb0 2023-07-14T19:23:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc019d710 con 0x7efdc0100740 2023-07-14T19:23:29.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc7244700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc019d8a0 con 0x7efdc00ffdb0 2023-07-14T19:23:29.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 0x7efdc019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:29.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 0x7efdc019d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38910/0 (socket says 172.21.15.133:38910) 2023-07-14T19:23:29.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc4fe0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc00febb0 0x7efdc0195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 -- 172.21.15.133:0/820244370 learned_addr learned my addr 172.21.15.133:0/820244370 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T19:23:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdbffff700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdc00ffdb0 0x7efdc0195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 -- 172.21.15.133:0/820244370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdc00ffdb0 msgr2=0x7efdc0195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdc00ffdb0 0x7efdc0195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 -- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc00febb0 msgr2=0x7efdc0195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc00febb0 0x7efdc0195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.600+0000 7efdc57e1700 1 -- 172.21.15.133:0/820244370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40095f0 con 0x7efdc0100740 2023-07-14T19:23:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.601+0000 7efdc57e1700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 0x7efdc019d000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efdb8007ee0 tx=0x7efdb800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:23:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.601+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8017070 con 0x7efdc0100740 2023-07-14T19:23:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.601+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc019db80 con 0x7efdc0100740 2023-07-14T19:23:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.601+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb800f040 con 0x7efdc0100740 2023-07-14T19:23:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.601+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8005610 con 0x7efdc0100740 2023-07-14T19:23:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.601+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc019e080 con 0x7efdc0100740 2023-07-14T19:23:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.602+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc010b520 con 0x7efdc0100740 2023-07-14T19:23:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.603+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efdb8004c40 con 0x7efdc0100740 2023-07-14T19:23:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.603+0000 7efdbdffb700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdb00a1150 0x7efdb00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.604+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3702+0+0 (secure 0 0 0) 0x7efdb80cadc0 con 0x7efdc0100740 2023-07-14T19:23:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.604+0000 7efdc4fe0700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdb00a1150 0x7efdb00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:29.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.604+0000 7efdc4fe0700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdb00a1150 0x7efdb00a3600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efdb400b5c0 tx=0x7efdb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:23:29.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.608+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efdb8095710 con 0x7efdc0100740 2023-07-14T19:23:29.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:29.808+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7efdc0060a80 con 0x7efdb00a1150 2023-07-14T19:23:29.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:29 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:23:29.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:29 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:23:29.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:29 smithi082 ceph-mon[127461]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:29.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:29 smithi082 ceph-mon[127461]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T19:23:29.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:29 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:23:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:29 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:23:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:29 smithi161 ceph-mon[129184]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:29.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:29 smithi161 ceph-mon[129184]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T19:23:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:29 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:23:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:29 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:23:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:29 smithi133 ceph-mon[128894]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-14T19:23:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:29 smithi133 ceph-mon[128894]: osdmap e27: 4 total, 4 up, 4 in 2023-07-14T19:23:30.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:30 smithi133 ceph-mon[128894]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:30.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:30.879 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:30.880 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:30.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:30 smithi082 ceph-mon[127461]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:30.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:30.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:30.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:30.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:30 smithi161 ceph-mon[129184]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:30.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:30.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:30.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:31.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:31 smithi161 ceph-mon[129184]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:31.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:31 smithi161 ceph-mon[129184]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T19:23:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:31 smithi133 ceph-mon[128894]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:31 smithi133 ceph-mon[128894]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T19:23:32.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:31 smithi082 ceph-mon[127461]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:32.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:31 smithi082 ceph-mon[127461]: osdmap e28: 4 total, 4 up, 4 in 2023-07-14T19:23:32.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:32 smithi161 ceph-mon[129184]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T19:23:32.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:32 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/169810330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]: dispatch 2023-07-14T19:23:32.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:32 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]: dispatch 2023-07-14T19:23:32.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:32 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]': finished 2023-07-14T19:23:32.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:32 smithi161 ceph-mon[129184]: osdmap e30: 5 total, 4 up, 5 in 2023-07-14T19:23:32.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:32 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:33.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:32 smithi133 ceph-mon[128894]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T19:23:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:32 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/169810330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]: dispatch 2023-07-14T19:23:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:32 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]: dispatch 2023-07-14T19:23:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:32 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]': finished 2023-07-14T19:23:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:32 smithi133 ceph-mon[128894]: osdmap e30: 5 total, 4 up, 5 in 2023-07-14T19:23:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:32 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:33.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:32 smithi082 ceph-mon[127461]: osdmap e29: 4 total, 4 up, 4 in 2023-07-14T19:23:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:32 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/169810330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]: dispatch 2023-07-14T19:23:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:32 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]: dispatch 2023-07-14T19:23:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:32 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f44de9db-95a0-4007-8acb-40ac3ed8161e"}]': finished 2023-07-14T19:23:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:32 smithi082 ceph-mon[127461]: osdmap e30: 5 total, 4 up, 5 in 2023-07-14T19:23:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:32 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:33.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:33 smithi161 ceph-mon[129184]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-07-14T19:23:33.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:33 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/2016367222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:34.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:33 smithi133 ceph-mon[128894]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-07-14T19:23:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:33 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/2016367222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:34.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:33 smithi082 ceph-mon[127461]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 145 KiB/s, 0 objects/s recovering 2023-07-14T19:23:34.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:33 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/2016367222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:35.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:35 smithi161 ceph-mon[129184]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-07-14T19:23:36.043 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:35 smithi133 ceph-mon[128894]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-07-14T19:23:36.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:35 smithi082 ceph-mon[127461]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-07-14T19:23:37.964 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:37 smithi133 ceph-mon[128894]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T19:23:37.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:37 smithi161 ceph-mon[129184]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T19:23:38.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:37 smithi082 ceph-mon[127461]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T19:23:38.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T19:23:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:38.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T19:23:38.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:39.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-14T19:23:39.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:39.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:39 smithi133 ceph-mon[128894]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:23:39.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:39 smithi133 ceph-mon[128894]: Deploying daemon osd.4 on smithi133 2023-07-14T19:23:39.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:39 smithi161 ceph-mon[129184]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:23:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:39 smithi161 ceph-mon[129184]: Deploying daemon osd.4 on smithi133 2023-07-14T19:23:40.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:39 smithi082 ceph-mon[127461]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:23:40.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:39 smithi082 ceph-mon[127461]: Deploying daemon osd.4 on smithi133 2023-07-14T19:23:41.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:41 smithi133 ceph-mon[128894]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-14T19:23:42.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:41 smithi082 ceph-mon[127461]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-14T19:23:42.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:41 smithi161 ceph-mon[129184]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-14T19:23:43.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:43 smithi133 ceph-mon[128894]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-07-14T19:23:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:43.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:43 smithi161 ceph-mon[129184]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-07-14T19:23:43.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:43.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:43.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:43.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:43 smithi082 ceph-mon[127461]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-07-14T19:23:43.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:43.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:43.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.415+0000 7efdbdffb700 1 -- 172.21.15.133:0/820244370 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efdc0060a80 con 0x7efdb00a1150 2023-07-14T19:23:45.417 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 4 on host 'smithi133' 2023-07-14T19:23:45.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdb00a1150 msgr2=0x7efdb00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:45.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdb00a1150 0x7efdb00a3600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efdb400b5c0 tx=0x7efdb4005fd0 comp rx=0 tx=0).stop 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 msgr2=0x7efdc019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 0x7efdc019d000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efdb8007ee0 tx=0x7efdb800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 shutdown_connections 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdc0100740 0x7efdc019d000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efdb00a1150 0x7efdb00a3600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc00febb0 0x7efdc0195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 --2- 172.21.15.133:0/820244370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdc00ffdb0 0x7efdc0195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:45.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 >> 172.21.15.133:0/820244370 conn(0x7efdc00fa1e0 msgr2=0x7efdc00fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:23:45.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 shutdown_connections 2023-07-14T19:23:45.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:23:45.418+0000 7efdc7244700 1 -- 172.21.15.133:0/820244370 wait complete. 2023-07-14T19:23:45.804 DEBUG:teuthology.orchestra.run.smithi133:osd.4> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.4.service 2023-07-14T19:23:45.806 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2023-07-14T19:23:45.807 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-14T19:23:45.837 INFO:journalctl@ceph.osd.4.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:23:45.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:45 smithi161 ceph-mon[129184]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:45.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:45.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:45.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:45.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:45 smithi133 ceph-mon[128894]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:46.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:45 smithi082 ceph-mon[127461]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:46.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:46.370 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:23:47.184 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:23:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4[139330]: 2023-07-14T19:23:46.957+0000 7f9a390e6640 -1 osd.4 0 log_to_monitors true 2023-07-14T19:23:47.953 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-14T19:23:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-14T19:23:47.954 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T19:23:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T19:23:47.954 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.014287 s, 734 MB/s 2023-07-14T19:23:47.955 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T19:23:47.989 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:47 smithi161 ceph-mon[129184]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:47.990 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:47 smithi161 ceph-mon[129184]: from='osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T19:23:47.990 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:47 smithi161 ceph-mon[129184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T19:23:48.150 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:47 smithi133 ceph-mon[128894]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:48.151 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:47 smithi133 ceph-mon[128894]: from='osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T19:23:48.151 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:47 smithi133 ceph-mon[128894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T19:23:48.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:47 smithi082 ceph-mon[127461]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:48.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:47 smithi082 ceph-mon[127461]: from='osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T19:23:48.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:47 smithi082 ceph-mon[127461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-14T19:23:48.426 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2023-07-14T19:23:48.979 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:23:49.184 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4[139330]: 2023-07-14T19:23:48.753+0000 7f9a2e75e700 -1 osd.4 0 waiting for initial osdmap 2023-07-14T19:23:49.184 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4[139330]: 2023-07-14T19:23:48.758+0000 7f9a27d4f700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:23:49.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T19:23:49.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:49.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:49.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:49.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T19:23:49.187 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T19:23:49.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:49.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:23:49.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:23:49.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.188 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:49.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:49.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:49.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:49.189 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:49.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:49.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:49.190 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-14T19:23:49.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: osdmap e31: 5 total, 4 up, 5 in 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:49.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:23:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:23:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:49.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:23:49.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:23:50.038 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi133 2023-07-14T19:23:50.038 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi133 to 8584M 2023-07-14T19:23:50.038 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:50.039 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:23:50.039 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T19:23:50.039 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:50.039 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:50.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi133 2023-07-14T19:23:50.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi133 to 8584M 2023-07-14T19:23:50.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:50.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:23:50.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T19:23:50.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:50.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:50.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 -- 172.21.15.161:0/2112168868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df0100ec0 msgr2=0x7f5df0101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:50.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 --2- 172.21.15.161:0/2112168868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df0100ec0 0x7f5df0101310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009990 tx=0x7f5dd8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:23:50.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 -- 172.21.15.161:0/2112168868 shutdown_connections 2023-07-14T19:23:50.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 --2- 172.21.15.161:0/2112168868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 0x7f5df0108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:50.196 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 --2- 172.21.15.161:0/2112168868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df00ffcc0 0x7f5df01000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:50.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 --2- 172.21.15.161:0/2112168868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df0100ec0 0x7f5df0101310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:50.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 -- 172.21.15.161:0/2112168868 >> 172.21.15.161:0/2112168868 conn(0x7f5df00fb2d0 msgr2=0x7f5df00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:23:50.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 -- 172.21.15.161:0/2112168868 shutdown_connections 2023-07-14T19:23:50.197 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.195+0000 7f5df64e5700 1 -- 172.21.15.161:0/2112168868 wait complete. 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 Processor -- start 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 -- start start 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 0x7f5df0197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df0100ec0 0x7f5df0198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 0x7f5df019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df019f760 con 0x7f5df0100ec0 2023-07-14T19:23:50.199 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df019f8a0 con 0x7f5df00ffcc0 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.196+0000 7f5df64e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df019fa60 con 0x7f5df0101850 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5deffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 0x7f5df0197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 0x7f5df019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5deffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 0x7f5df0197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.161:53454/0 (socket says 172.21.15.161:53454) 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5deffff700 1 -- 172.21.15.161:0/3653683102 learned_addr learned my addr 172.21.15.161:0/3653683102 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5def7fe700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df0100ec0 0x7f5df0198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:50.200 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 -- 172.21.15.161:0/3653683102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 msgr2=0x7f5df0197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 0x7f5df0197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 -- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df0100ec0 msgr2=0x7f5df0198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df0100ec0 0x7f5df0198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 -- 172.21.15.161:0/3653683102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8009c40 con 0x7f5df0101850 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5def7fe700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df0100ec0 0x7f5df0198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5deffff700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 0x7f5df0197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df4a82700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 0x7f5df019f190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5de800efb0 tx=0x7f5de800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:23:50.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800ca50 con 0x7f5df0101850 2023-07-14T19:23:50.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd80095f0 con 0x7f5df0101850 2023-07-14T19:23:50.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de800f040 con 0x7f5df0101850 2023-07-14T19:23:50.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8004660 con 0x7f5df0101850 2023-07-14T19:23:50.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.197+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df01a0100 con 0x7f5df0101850 2023-07-14T19:23:50.202 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.198+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df010b480 con 0x7f5df0101850 2023-07-14T19:23:50.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.199+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5de80047c0 con 0x7f5df0101850 2023-07-14T19:23:50.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.200+0000 7f5ded7fa700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5ddc0a1150 0x7f5ddc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:23:50.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.200+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f5de80cb7b0 con 0x7f5df0101850 2023-07-14T19:23:50.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.200+0000 7f5deffff700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5ddc0a1150 0x7f5ddc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:23:50.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.201+0000 7f5deffff700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5ddc0a1150 0x7f5ddc0a3600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5df01990f0 tx=0x7f5de0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:23:50.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.204+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5de8096000 con 0x7f5df0101850 2023-07-14T19:23:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi133 2023-07-14T19:23:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi133 to 8584M 2023-07-14T19:23:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:23:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: osdmap e32: 5 total, 4 up, 5 in 2023-07-14T19:23:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:50.396 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:23:50.395+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5df0107860 con 0x7f5ddc0a1150 2023-07-14T19:23:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:23:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:23:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] boot 2023-07-14T19:23:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: osdmap e33: 5 total, 5 up, 5 in 2023-07-14T19:23:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:51.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:23:51.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:23:51.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:51.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] boot 2023-07-14T19:23:51.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: osdmap e33: 5 total, 5 up, 5 in 2023-07-14T19:23:51.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:51.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:51.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:51.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:51.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:23:51.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:23:51.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: osd.4 [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] boot 2023-07-14T19:23:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: osdmap e33: 5 total, 5 up, 5 in 2023-07-14T19:23:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:23:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:23:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:23:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:23:51.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:51 smithi161 ceph-mon[129184]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:51.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:51 smithi161 ceph-mon[129184]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:51.987 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:51 smithi161 ceph-mon[129184]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T19:23:52.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:51 smithi133 ceph-mon[128894]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:51 smithi133 ceph-mon[128894]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:51 smithi133 ceph-mon[128894]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T19:23:52.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:51 smithi082 ceph-mon[127461]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:23:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:51 smithi082 ceph-mon[127461]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-07-14T19:23:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:51 smithi082 ceph-mon[127461]: osdmap e34: 5 total, 5 up, 5 in 2023-07-14T19:23:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/1155403169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]: dispatch 2023-07-14T19:23:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]: dispatch 2023-07-14T19:23:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]': finished 2023-07-14T19:23:53.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: osdmap e35: 6 total, 5 up, 6 in 2023-07-14T19:23:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:23:53.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:53 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/3361899020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:53.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:53.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/1155403169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]: dispatch 2023-07-14T19:23:53.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]: dispatch 2023-07-14T19:23:53.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]': finished 2023-07-14T19:23:53.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: osdmap e35: 6 total, 5 up, 6 in 2023-07-14T19:23:53.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:23:53.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:53 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/3361899020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:53.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:53.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/1155403169' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]: dispatch 2023-07-14T19:23:53.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]: dispatch 2023-07-14T19:23:53.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "351f3277-08c4-46e3-b1ae-1f78290f556a"}]': finished 2023-07-14T19:23:53.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: osdmap e35: 6 total, 5 up, 6 in 2023-07-14T19:23:53.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:23:53.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:53 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/3361899020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:23:55.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:55 smithi161 ceph-mon[129184]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:56.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:55 smithi133 ceph-mon[128894]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:56.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:55 smithi082 ceph-mon[127461]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:57.955 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:57 smithi161 ceph-mon[129184]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:58.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:57 smithi133 ceph-mon[128894]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:58.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:57 smithi082 ceph-mon[127461]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:59.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:59 smithi161 ceph-mon[129184]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:23:59.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T19:23:59.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:23:59 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:59 smithi133 ceph-mon[128894]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T19:24:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:23:59 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:59 smithi082 ceph-mon[127461]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-14T19:24:00.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:23:59 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:00 smithi133 ceph-mon[128894]: Deploying daemon osd.5 on smithi161 2023-07-14T19:24:01.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:00 smithi082 ceph-mon[127461]: Deploying daemon osd.5 on smithi161 2023-07-14T19:24:01.221 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:00 smithi161 ceph-mon[129184]: Deploying daemon osd.5 on smithi161 2023-07-14T19:24:01.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:01 smithi161 ceph-mon[129184]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:02.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:01 smithi133 ceph-mon[128894]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:02.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:01 smithi082 ceph-mon[127461]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:03.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:03 smithi161 ceph-mon[129184]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:03.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:03 smithi133 ceph-mon[128894]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:03.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:03 smithi082 ceph-mon[127461]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:04.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:04.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:04.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:05.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:05.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:05.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:05 smithi133 ceph-mon[128894]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:06.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:05 smithi082 ceph-mon[127461]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:05 smithi161 ceph-mon[129184]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:06.806 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.804+0000 7f5ded7fa700 1 -- 172.21.15.161:0/3653683102 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5df0107860 con 0x7f5ddc0a1150 2023-07-14T19:24:06.806 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2023-07-14T19:24:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.806+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5ddc0a1150 msgr2=0x7f5ddc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5ddc0a1150 0x7f5ddc0a3600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5df01990f0 tx=0x7f5de0009260 comp rx=0 tx=0).stop 2023-07-14T19:24:06.808 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 msgr2=0x7f5df019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 0x7f5df019f190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5de800efb0 tx=0x7f5de800c3a0 comp rx=0 tx=0).stop 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 shutdown_connections 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5df0101850 0x7f5df019f190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df00ffcc0 0x7f5df0197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5ddc0a1150 0x7f5ddc0a3600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 --2- 172.21.15.161:0/3653683102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5df0100ec0 0x7f5df0198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 >> 172.21.15.161:0/3653683102 conn(0x7f5df00fb2d0 msgr2=0x7f5df00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:24:06.809 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 shutdown_connections 2023-07-14T19:24:06.810 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:06.807+0000 7f5df64e5700 1 -- 172.21.15.161:0/3653683102 wait complete. 2023-07-14T19:24:07.270 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.5.service 2023-07-14T19:24:07.273 INFO:tasks.cephadm:Deploying osd.6 on smithi082 with /dev/vg_nvme/lv_2... 2023-07-14T19:24:07.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T19:24:07.304 INFO:journalctl@ceph.osd.5.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:24:07.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:07.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:07.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:07 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:07.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:07.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:07.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:07.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:07.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:07 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:07.841 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:24:08.103 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:24:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:24:07.816+0000 7f88dee31640 -1 osd.5 0 log_to_monitors true 2023-07-14T19:24:08.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:08 smithi082 ceph-mon[127461]: from='osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T19:24:08.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:08 smithi082 ceph-mon[127461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T19:24:08.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:08 smithi161 ceph-mon[129184]: from='osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T19:24:08.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:08 smithi161 ceph-mon[129184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T19:24:08.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:08 smithi133 ceph-mon[128894]: from='osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T19:24:08.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:08 smithi133 ceph-mon[128894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-14T19:24:09.337 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T19:24:09.337 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T19:24:09.337 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-07-14T19:24:09.337 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-07-14T19:24:09.338 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111725 s, 939 MB/s 2023-07-14T19:24:09.338 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-07-14T19:24:09.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:09 smithi082 ceph-mon[127461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T19:24:09.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:09 smithi082 ceph-mon[127461]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T19:24:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:09 smithi082 ceph-mon[127461]: from='osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:24:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:09 smithi082 ceph-mon[127461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:24:09.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:09 smithi082 ceph-mon[127461]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:09.741 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:24:09.499+0000 7f88d44a9700 -1 osd.5 0 waiting for initial osdmap 2023-07-14T19:24:09.742 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:24:09.505+0000 7f88cc297700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:24:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-mon[129184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T19:24:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-mon[129184]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T19:24:09.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:09.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-mon[129184]: from='osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:24:09.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-mon[129184]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:24:09.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:09 smithi161 ceph-mon[129184]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:09.787 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2023-07-14T19:24:09.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:09 smithi133 ceph-mon[128894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-14T19:24:09.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:09 smithi133 ceph-mon[128894]: osdmap e36: 6 total, 5 up, 6 in 2023-07-14T19:24:09.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:09.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:09 smithi133 ceph-mon[128894]: from='osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:24:09.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:09 smithi133 ceph-mon[128894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:24:09.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:09 smithi133 ceph-mon[128894]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:10.350 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:24:10.890 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi161 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.891 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:10.892 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:10.892 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi161 to 8584M 2023-07-14T19:24:10.892 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.892 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:10.892 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:10.892 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:10 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:24:10.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T19:24:10.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:10.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:10.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi161 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi161 to 8584M 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:10.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:10.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:10 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:24:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: osdmap e37: 6 total, 5 up, 6 in 2023-07-14T19:24:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi161 2023-07-14T19:24:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi161 to 8584M 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:10.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:10 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/2737575482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 msgr2=0x7fe3e4100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/2737575482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4100270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009990 tx=0x7fe3e0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/2737575482 shutdown_connections 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/2737575482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e41007b0 0x7fe3e4107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/2737575482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e40ff0e0 0x7fe3e40ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/2737575482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4100270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/2737575482 >> 172.21.15.82:0/2737575482 conn(0x7fe3e40f9990 msgr2=0x7fe3e40fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/2737575482 shutdown_connections 2023-07-14T19:24:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.542+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/2737575482 wait complete. 2023-07-14T19:24:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 Processor -- start 2023-07-14T19:24:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 -- start start 2023-07-14T19:24:11.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e40ff0e0 0x7fe3e41094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:11.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4107b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41007b0 0x7fe3e4108050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41087d0 con 0x7fe3e40ffe00 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4108940 con 0x7fe3e40ff0e0 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ec3b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e419f9f0 con 0x7fe3e41007b0 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40042/0 (socket says 172.21.15.82:40042) 2023-07-14T19:24:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 -- 172.21.15.82:0/1876806967 learned_addr learned my addr 172.21.15.82:0/1876806967 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ea954700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41007b0 0x7fe3e4108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ea153700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e40ff0e0 0x7fe3e41094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 -- 172.21.15.82:0/1876806967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41007b0 msgr2=0x7fe3e4108050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41007b0 0x7fe3e4108050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 -- 172.21.15.82:0/1876806967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e40ff0e0 msgr2=0x7fe3e41094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e40ff0e0 0x7fe3e41094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3e9952700 1 -- 172.21.15.82:0/1876806967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00095f0 con 0x7fe3e40ffe00 2023-07-14T19:24:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.543+0000 7fe3ea153700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e40ff0e0 0x7fe3e41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.544+0000 7fe3e9952700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4107b10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00095c0 tx=0x7fe3e000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.544+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e001d070 con 0x7fe3e40ffe00 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.544+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e0022470 con 0x7fe3e40ffe00 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.544+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e419fc70 con 0x7fe3e40ffe00 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.544+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0017650 con 0x7fe3e40ffe00 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.544+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e41a0080 con 0x7fe3e40ffe00 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.546+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e41061d0 con 0x7fe3e40ffe00 2023-07-14T19:24:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.547+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe3e000fad0 con 0x7fe3e40ffe00 2023-07-14T19:24:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.547+0000 7fe3db7fe700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3d00a1150 0x7fe3d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.547+0000 7fe3ea153700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3d00a1150 0x7fe3d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.547+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fe3e00ccd10 con 0x7fe3e40ffe00 2023-07-14T19:24:11.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.548+0000 7fe3ea153700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3d00a1150 0x7fe3d00a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005950 tx=0x7fe3d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:24:11.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.551+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe3e00974b0 con 0x7fe3e40ffe00 2023-07-14T19:24:11.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:24:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:24:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] boot 2023-07-14T19:24:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: osdmap e38: 6 total, 6 up, 6 in 2023-07-14T19:24:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:11.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:11 smithi082 ceph-mon[127461]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:11.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:11.748+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3e4060a80 con 0x7fe3d00a1150 2023-07-14T19:24:11.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:24:11.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:24:11.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:11.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] boot 2023-07-14T19:24:11.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: osdmap e38: 6 total, 6 up, 6 in 2023-07-14T19:24:11.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:11.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:11 smithi133 ceph-mon[128894]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:11.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:24:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:24:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: osd.5 [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] boot 2023-07-14T19:24:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: osdmap e38: 6 total, 6 up, 6 in 2023-07-14T19:24:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:24:11.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:11 smithi161 ceph-mon[129184]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-07-14T19:24:12.814 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:12 smithi082 ceph-mon[127461]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T19:24:12.815 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:12 smithi082 ceph-mon[127461]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:12.815 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:12.815 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:12.815 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:12 smithi133 ceph-mon[128894]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T19:24:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:12 smithi133 ceph-mon[128894]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:12.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:12.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:12.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:12 smithi161 ceph-mon[129184]: osdmap e39: 6 total, 6 up, 6 in 2023-07-14T19:24:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:12 smithi161 ceph-mon[129184]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:12.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:13.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:13 smithi082 ceph-mon[127461]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T19:24:13.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:13 smithi082 ceph-mon[127461]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:13.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:13 smithi133 ceph-mon[128894]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T19:24:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:13 smithi133 ceph-mon[128894]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:13.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:13 smithi161 ceph-mon[129184]: osdmap e40: 6 total, 6 up, 6 in 2023-07-14T19:24:13.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:13 smithi161 ceph-mon[129184]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:14.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:14 smithi133 ceph-mon[128894]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T19:24:14.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:14 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1366912700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69302d50-5480-44c9-815d-c580832c5198"}]: dispatch 2023-07-14T19:24:14.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:14 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1366912700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69302d50-5480-44c9-815d-c580832c5198"}]': finished 2023-07-14T19:24:14.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:14 smithi133 ceph-mon[128894]: osdmap e42: 7 total, 6 up, 7 in 2023-07-14T19:24:14.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:14 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:14.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:14 smithi082 ceph-mon[127461]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T19:24:14.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:14 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1366912700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69302d50-5480-44c9-815d-c580832c5198"}]: dispatch 2023-07-14T19:24:14.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:14 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1366912700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69302d50-5480-44c9-815d-c580832c5198"}]': finished 2023-07-14T19:24:14.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:14 smithi082 ceph-mon[127461]: osdmap e42: 7 total, 6 up, 7 in 2023-07-14T19:24:14.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:14 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:14.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:14 smithi161 ceph-mon[129184]: osdmap e41: 6 total, 6 up, 6 in 2023-07-14T19:24:14.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:14 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1366912700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69302d50-5480-44c9-815d-c580832c5198"}]: dispatch 2023-07-14T19:24:14.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:14 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1366912700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69302d50-5480-44c9-815d-c580832c5198"}]': finished 2023-07-14T19:24:14.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:14 smithi161 ceph-mon[129184]: osdmap e42: 7 total, 6 up, 7 in 2023-07-14T19:24:14.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:14 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:15.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:15 smithi133 ceph-mon[128894]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:15.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:15 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3782365021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:15.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:15 smithi082 ceph-mon[127461]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:15.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:15 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3782365021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:15.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:15 smithi161 ceph-mon[129184]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:15.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:15 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3782365021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:17.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:17 smithi161 ceph-mon[129184]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:18.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:17 smithi133 ceph-mon[128894]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:18.195 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:17 smithi082 ceph-mon[127461]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:19.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:19 smithi161 ceph-mon[129184]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-14T19:24:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:19 smithi133 ceph-mon[128894]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-14T19:24:20.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:19 smithi082 ceph-mon[127461]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-14T19:24:20.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:20 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T19:24:20.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:20 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:21.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:20 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T19:24:21.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:20 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:21.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:20 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-14T19:24:21.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:20 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:21 smithi082 ceph-mon[127461]: Deploying daemon osd.6 on smithi082 2023-07-14T19:24:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:21 smithi082 ceph-mon[127461]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:24:22.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:21 smithi133 ceph-mon[128894]: Deploying daemon osd.6 on smithi082 2023-07-14T19:24:22.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:21 smithi133 ceph-mon[128894]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:24:22.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:21 smithi161 ceph-mon[129184]: Deploying daemon osd.6 on smithi082 2023-07-14T19:24:22.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:21 smithi161 ceph-mon[129184]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:24:23.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:23 smithi082 ceph-mon[127461]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-14T19:24:23.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:23 smithi161 ceph-mon[129184]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-14T19:24:23.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:23 smithi133 ceph-mon[128894]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-14T19:24:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:25 smithi082 ceph-mon[127461]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-07-14T19:24:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:26.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:25 smithi133 ceph-mon[128894]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-07-14T19:24:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:26.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:25 smithi161 ceph-mon[129184]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-07-14T19:24:26.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:26.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:26.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:27.549 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.549+0000 7fe3db7fe700 1 -- 172.21.15.82:0/1876806967 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe3e4060a80 con 0x7fe3d00a1150 2023-07-14T19:24:27.550 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 6 on host 'smithi082' 2023-07-14T19:24:27.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3d00a1150 msgr2=0x7fe3d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3d00a1150 0x7fe3d00a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005950 tx=0x7fe3d40058e0 comp rx=0 tx=0).stop 2023-07-14T19:24:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 msgr2=0x7fe3e4107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4107b10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00095c0 tx=0x7fe3e000f690 comp rx=0 tx=0).stop 2023-07-14T19:24:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 shutdown_connections 2023-07-14T19:24:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3e41007b0 0x7fe3e4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:27.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e40ff0e0 0x7fe3e41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3d00a1150 0x7fe3d00a3600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 --2- 172.21.15.82:0/1876806967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3e40ffe00 0x7fe3e4107b10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 >> 172.21.15.82:0/1876806967 conn(0x7fe3e40f9990 msgr2=0x7fe3e4103310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:24:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 shutdown_connections 2023-07-14T19:24:27.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:24:27.551+0000 7fe3ec3b7700 1 -- 172.21.15.82:0/1876806967 wait complete. 2023-07-14T19:24:27.936 DEBUG:teuthology.orchestra.run.smithi082:osd.6> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.6.service 2023-07-14T19:24:27.938 INFO:tasks.cephadm:Deploying osd.7 on smithi133 with /dev/vg_nvme/lv_2... 2023-07-14T19:24:27.938 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T19:24:27.968 INFO:journalctl@ceph.osd.6.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:24:28.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:28 smithi082 ceph-mon[127461]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:24:28.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:28 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:28 smithi133 ceph-mon[128894]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:24:28.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:28 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:28 smithi161 ceph-mon[129184]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:24:28.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:28 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:28.517 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:24:29.102 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:24:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:24:28.748+0000 7f2fe9bb3640 -1 osd.6 0 log_to_monitors true 2023-07-14T19:24:29.425 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:29 smithi133 ceph-mon[128894]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T19:24:29.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:29 smithi082 ceph-mon[127461]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T19:24:29.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:29 smithi161 ceph-mon[129184]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-14T19:24:29.908 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T19:24:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T19:24:29.909 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T19:24:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T19:24:29.909 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0144627 s, 725 MB/s 2023-07-14T19:24:29.909 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T19:24:30.303 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:30 smithi133 ceph-mon[128894]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:24:30.304 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:30 smithi133 ceph-mon[128894]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T19:24:30.305 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:30 smithi133 ceph-mon[128894]: osdmap e43: 7 total, 6 up, 7 in 2023-07-14T19:24:30.305 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:30.306 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:30 smithi133 ceph-mon[128894]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:24:30.358 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2023-07-14T19:24:30.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-mon[127461]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:24:30.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-mon[127461]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T19:24:30.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-mon[127461]: osdmap e43: 7 total, 6 up, 7 in 2023-07-14T19:24:30.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:30.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-mon[127461]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:24:30.465 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:24:30.114+0000 7f2fdf22b700 -1 osd.6 0 waiting for initial osdmap 2023-07-14T19:24:30.465 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:24:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:24:30.125+0000 7f2fd7019700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:24:30.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:30 smithi161 ceph-mon[129184]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:24:30.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:30 smithi161 ceph-mon[129184]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-14T19:24:30.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:30 smithi161 ceph-mon[129184]: osdmap e43: 7 total, 6 up, 7 in 2023-07-14T19:24:30.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:30.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:30 smithi161 ceph-mon[129184]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:24:30.913 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:24:31.410 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:24:31.411 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T19:24:31.411 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:31.411 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:31.411 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.411 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:31.412 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.413 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:31.413 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:31.413 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:31.413 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:24:31.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T19:24:31.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:31.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:31.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:31.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:31.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:31.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:31.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:24:31.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: osdmap e44: 7 total, 6 up, 7 in 2023-07-14T19:24:31.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:31.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:31.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:31.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:31.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:31.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:31.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:32.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 -- 172.21.15.133:0/2274245189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e00100740 msgr2=0x7f1e00107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:32.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 --2- 172.21.15.133:0/2274245189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e00100740 0x7f1e00107000 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b0a0 tx=0x7f1df800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:24:32.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 -- 172.21.15.133:0/2274245189 shutdown_connections 2023-07-14T19:24:32.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 --2- 172.21.15.133:0/2274245189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e00100740 0x7f1e00107000 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:32.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 --2- 172.21.15.133:0/2274245189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e000febb0 0x7f1e000fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:32.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 --2- 172.21.15.133:0/2274245189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e000ffdb0 0x7f1e00100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:32.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 -- 172.21.15.133:0/2274245189 >> 172.21.15.133:0/2274245189 conn(0x7f1e000fa1e0 msgr2=0x7f1e000fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:24:32.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 -- 172.21.15.133:0/2274245189 shutdown_connections 2023-07-14T19:24:32.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.112+0000 7f1e08334700 1 -- 172.21.15.133:0/2274245189 wait complete. 2023-07-14T19:24:32.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 Processor -- start 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 -- start start 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e000febb0 0x7f1e00194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 0x7f1e00194ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e00100740 0x7f1e0019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019c510 con 0x7f1e00100740 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019c650 con 0x7f1e000ffdb0 2023-07-14T19:24:32.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e08334700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0019c7e0 con 0x7f1e000febb0 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e058cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 0x7f1e00194ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e058cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 0x7f1e00194ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39012/0 (socket says 172.21.15.133:39012) 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e058cf700 1 -- 172.21.15.133:0/1112467986 learned_addr learned my addr 172.21.15.133:0/1112467986 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e068d1700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e00100740 0x7f1e0019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.113+0000 7f1e060d0700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e000febb0 0x7f1e00194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e058cf700 1 -- 172.21.15.133:0/1112467986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e000febb0 msgr2=0x7f1e00194980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e058cf700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e000febb0 0x7f1e00194980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:32.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e058cf700 1 -- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e00100740 msgr2=0x7f1e0019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e058cf700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e00100740 0x7f1e0019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e058cf700 1 -- 172.21.15.133:0/1112467986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009d00 con 0x7f1e000ffdb0 2023-07-14T19:24:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e058cf700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 0x7f1e00194ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00d950 tx=0x7f1dfc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:24:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc009940 con 0x7f1e000ffdb0 2023-07-14T19:24:32.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dfc010460 con 0x7f1e000ffdb0 2023-07-14T19:24:32.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0019cac0 con 0x7f1e000ffdb0 2023-07-14T19:24:32.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00f600 con 0x7f1e000ffdb0 2023-07-14T19:24:32.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.114+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0019cfc0 con 0x7f1e000ffdb0 2023-07-14T19:24:32.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.115+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e00106110 con 0x7f1e000ffdb0 2023-07-14T19:24:32.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.116+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1dfc0105d0 con 0x7f1e000ffdb0 2023-07-14T19:24:32.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.116+0000 7f1df77fe700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1dec0a1150 0x7f1dec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:32.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.117+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f1dfc0cba60 con 0x7f1e000ffdb0 2023-07-14T19:24:32.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.117+0000 7f1e060d0700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1dec0a1150 0x7f1dec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:32.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.117+0000 7f1e060d0700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1dec0a1150 0x7f1dec0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1df0005950 tx=0x7f1df00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:24:32.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.120+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1dfc095ec0 con 0x7f1e000ffdb0 2023-07-14T19:24:32.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi082 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi082 to 5722M 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] boot 2023-07-14T19:24:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: osdmap e45: 7 total, 7 up, 7 in 2023-07-14T19:24:32.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:32 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:32.313+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e00060a80 con 0x7f1dec0a1150 2023-07-14T19:24:32.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:24:32.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:24:32.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi082 2023-07-14T19:24:32.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi082 to 5722M 2023-07-14T19:24:32.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:32.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:32.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] boot 2023-07-14T19:24:32.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: osdmap e45: 7 total, 7 up, 7 in 2023-07-14T19:24:32.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:32 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:32.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi082 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi082 to 5722M 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: osd.6 [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] boot 2023-07-14T19:24:32.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: osdmap e45: 7 total, 7 up, 7 in 2023-07-14T19:24:32.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:32 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:24:33.418 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:33 smithi133 ceph-mon[128894]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:33.419 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:33.419 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:33.419 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:33.419 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:33 smithi133 ceph-mon[128894]: osdmap e46: 7 total, 7 up, 7 in 2023-07-14T19:24:33.419 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:33 smithi133 ceph-mon[128894]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:33.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:33 smithi082 ceph-mon[127461]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:33.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:33 smithi082 ceph-mon[127461]: osdmap e46: 7 total, 7 up, 7 in 2023-07-14T19:24:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:33 smithi082 ceph-mon[127461]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:33.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:33 smithi161 ceph-mon[129184]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:33 smithi161 ceph-mon[129184]: osdmap e46: 7 total, 7 up, 7 in 2023-07-14T19:24:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:33 smithi161 ceph-mon[129184]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:34.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:34 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/1038464164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]: dispatch 2023-07-14T19:24:34.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:34 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]: dispatch 2023-07-14T19:24:34.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:34 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]': finished 2023-07-14T19:24:34.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:34 smithi082 ceph-mon[127461]: osdmap e47: 8 total, 7 up, 8 in 2023-07-14T19:24:34.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:34 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:34 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/1038464164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]: dispatch 2023-07-14T19:24:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:34 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]: dispatch 2023-07-14T19:24:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:34 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]': finished 2023-07-14T19:24:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:34 smithi161 ceph-mon[129184]: osdmap e47: 8 total, 7 up, 8 in 2023-07-14T19:24:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:34 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:35.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:34 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/1038464164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]: dispatch 2023-07-14T19:24:35.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:34 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]: dispatch 2023-07-14T19:24:35.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:34 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4fa7360-353c-455e-a2d5-4fe907ce87ab"}]': finished 2023-07-14T19:24:35.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:34 smithi133 ceph-mon[128894]: osdmap e47: 8 total, 7 up, 8 in 2023-07-14T19:24:35.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:34 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:35.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:35 smithi082 ceph-mon[127461]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:35.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:35 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/4216269505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:35.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:35 smithi161 ceph-mon[129184]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:35.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:35 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/4216269505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:35 smithi133 ceph-mon[128894]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:35 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/4216269505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:37.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:37 smithi082 ceph-mon[127461]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:37.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:37 smithi161 ceph-mon[129184]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:38.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:37 smithi133 ceph-mon[128894]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:39.971 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:39 smithi133 ceph-mon[128894]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:39.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:39 smithi161 ceph-mon[129184]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:40.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:39 smithi082 ceph-mon[127461]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:41.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:41 smithi133 ceph-mon[128894]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:41.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T19:24:41.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:41.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:41 smithi133 ceph-mon[128894]: Deploying daemon osd.7 on smithi133 2023-07-14T19:24:41.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:41 smithi161 ceph-mon[129184]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:41.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T19:24:41.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:41.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:41 smithi161 ceph-mon[129184]: Deploying daemon osd.7 on smithi133 2023-07-14T19:24:42.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:41 smithi082 ceph-mon[127461]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:42.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-14T19:24:42.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:42.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:41 smithi082 ceph-mon[127461]: Deploying daemon osd.7 on smithi133 2023-07-14T19:24:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:43 smithi133 ceph-mon[128894]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:43.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:43 smithi082 ceph-mon[127461]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:43.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:43 smithi161 ceph-mon[129184]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:46.089 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:45 smithi133 ceph-mon[128894]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:46.089 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:46.089 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:46.089 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:46.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:45 smithi082 ceph-mon[127461]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:46.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:46.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:46.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:46.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:45 smithi161 ceph-mon[129184]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:46.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:46.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:46.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:47.866 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:47 smithi133 ceph-mon[128894]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:47.866 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:47.866 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:47.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.920+0000 7f1df77fe700 1 -- 172.21.15.133:0/1112467986 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1e00060a80 con 0x7f1dec0a1150 2023-07-14T19:24:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 7 on host 'smithi133' 2023-07-14T19:24:47.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1dec0a1150 msgr2=0x7f1dec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:47.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1dec0a1150 0x7f1dec0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1df0005950 tx=0x7f1df00058e0 comp rx=0 tx=0).stop 2023-07-14T19:24:47.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 msgr2=0x7f1e00194ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:47.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 0x7f1e00194ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00d950 tx=0x7f1dfc00dc60 comp rx=0 tx=0).stop 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 shutdown_connections 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1dec0a1150 0x7f1dec0a3600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e00100740 0x7f1e0019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e000febb0 0x7f1e00194980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 --2- 172.21.15.133:0/1112467986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e000ffdb0 0x7f1e00194ec0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 >> 172.21.15.133:0/1112467986 conn(0x7f1e000fa1e0 msgr2=0x7f1e00103250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:24:47.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 shutdown_connections 2023-07-14T19:24:47.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:24:47.923+0000 7f1e08334700 1 -- 172.21.15.133:0/1112467986 wait complete. 2023-07-14T19:24:48.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:47 smithi082 ceph-mon[127461]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:48.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:48.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:47 smithi161 ceph-mon[129184]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:48.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:48.312 DEBUG:teuthology.orchestra.run.smithi133:osd.7> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.7.service 2023-07-14T19:24:48.315 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2023-07-14T19:24:48.315 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-14T19:24:48.343 INFO:journalctl@ceph.osd.7.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:24:48.890 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:24:49.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:49.120 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:48 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:49.124 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:49.124 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:48 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:49.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:49.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:48 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:49.404 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:24:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:24:49.252+0000 7fb88e879640 -1 osd.7 0 log_to_monitors true 2023-07-14T19:24:49.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:49 smithi133 ceph-mon[128894]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:49.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:49 smithi133 ceph-mon[128894]: from='osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T19:24:49.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:49 smithi133 ceph-mon[128894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T19:24:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:49 smithi082 ceph-mon[127461]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:49 smithi082 ceph-mon[127461]: from='osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T19:24:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:49 smithi082 ceph-mon[127461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T19:24:50.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:49 smithi161 ceph-mon[129184]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:49 smithi161 ceph-mon[129184]: from='osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T19:24:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:49 smithi161 ceph-mon[129184]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-14T19:24:50.428 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-14T19:24:50.428 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-14T19:24:50.428 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T19:24:50.428 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T19:24:50.428 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0147014 s, 713 MB/s 2023-07-14T19:24:50.428 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T19:24:50.834 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2023-07-14T19:24:51.183 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:24:50.898+0000 7fb883ef1700 -1 osd.7 0 waiting for initial osdmap 2023-07-14T19:24:51.184 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:24:50.906+0000 7fb87f4e6700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:24:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T19:24:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: osdmap e48: 8 total, 7 up, 8 in 2023-07-14T19:24:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:24:51.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:51.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:51.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:51.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:51.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:51.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:50 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T19:24:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: osdmap e48: 8 total, 7 up, 8 in 2023-07-14T19:24:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:24:51.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:24:51.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:51.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:50 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-14T19:24:51.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: osdmap e48: 8 total, 7 up, 8 in 2023-07-14T19:24:51.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:51.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:24:51.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:24:51.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:24:51.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:51.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:24:51.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:50 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:24:51.389 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:24:52.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi133 2023-07-14T19:24:52.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi133 to 5722M 2023-07-14T19:24:52.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:52.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:24:52.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T19:24:52.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:52.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:51 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:52.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi133 2023-07-14T19:24:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi133 to 5722M 2023-07-14T19:24:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:24:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T19:24:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:52.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:51 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:52.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi133 2023-07-14T19:24:52.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi133 to 5722M 2023-07-14T19:24:52.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:24:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: osdmap e49: 8 total, 7 up, 8 in 2023-07-14T19:24:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:51 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:52.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 -- 172.21.15.161:0/3699348995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 msgr2=0x7fc71c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:52.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 --2- 172.21.15.161:0/3699348995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c1067e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc714009f90 tx=0x7fc71400b330 comp rx=0 tx=0).stop 2023-07-14T19:24:52.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 -- 172.21.15.161:0/3699348995 shutdown_connections 2023-07-14T19:24:52.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 --2- 172.21.15.161:0/3699348995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c1067e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:52.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 --2- 172.21.15.161:0/3699348995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 0x7fc71c0fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:52.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 --2- 172.21.15.161:0/3699348995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 0x7fc71c0ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:52.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 -- 172.21.15.161:0/3699348995 >> 172.21.15.161:0/3699348995 conn(0x7fc71c0f99c0 msgr2=0x7fc71c0fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:24:52.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.617+0000 7fc72142e700 1 -- 172.21.15.161:0/3699348995 shutdown_connections 2023-07-14T19:24:52.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.618+0000 7fc72142e700 1 -- 172.21.15.161:0/3699348995 wait complete. 2023-07-14T19:24:52.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 Processor -- start 2023-07-14T19:24:52.622 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 -- start start 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 0x7fc71c10b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 0x7fc71c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c19d5a0 con 0x7fc71c0ff590 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c19d710 con 0x7fc71c0fe390 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc72142e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c19d8d0 con 0x7fc71c0fff20 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:52.623 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60322/0 (socket says 172.21.15.161:60322) 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 -- 172.21.15.161:0/2125171526 learned_addr learned my addr 172.21.15.161:0/2125171526 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71a7fc700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 0x7fc71c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71affd700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 0x7fc71c10b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 -- 172.21.15.161:0/2125171526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 msgr2=0x7fc71c10b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 0x7fc71c10b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 -- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 msgr2=0x7fc71c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:24:52.624 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 0x7fc71c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71b7fe700 1 -- 172.21.15.161:0/2125171526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100095f0 con 0x7fc71c0fff20 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71a7fc700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 0x7fc71c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.619+0000 7fc71affd700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 0x7fc71c10b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.620+0000 7fc71b7fe700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c19ce00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc714009f60 tx=0x7fc714004d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.620+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400e040 con 0x7fc71c0fff20 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.620+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c009520 con 0x7fc71c0fff20 2023-07-14T19:24:52.625 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.620+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7140093a0 con 0x7fc71c0fff20 2023-07-14T19:24:52.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.620+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140052a0 con 0x7fc71c0fff20 2023-07-14T19:24:52.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.620+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc714009c40 con 0x7fc71c0fff20 2023-07-14T19:24:52.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.621+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c104f70 con 0x7fc71c0fff20 2023-07-14T19:24:52.626 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.622+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc71401d470 con 0x7fc71c0fff20 2023-07-14T19:24:52.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.622+0000 7fc703fff700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc7040a1150 0x7fc7040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:24:52.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.622+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fc7140cd390 con 0x7fc71c0fff20 2023-07-14T19:24:52.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.623+0000 7fc71affd700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc7040a1150 0x7fc7040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:24:52.627 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.623+0000 7fc71affd700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc7040a1150 0x7fc7040a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ff3f0 tx=0x7fc710009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:24:52.628 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.627+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc7140976b0 con 0x7fc71c0fff20 2023-07-14T19:24:52.823 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:24:52.822+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc71c060a80 con 0x7fc7040a1150 2023-07-14T19:24:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:24:53.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:24:53.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:53.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] boot 2023-07-14T19:24:53.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: osdmap e50: 8 total, 8 up, 8 in 2023-07-14T19:24:53.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:53.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: osdmap e51: 8 total, 8 up, 8 in 2023-07-14T19:24:53.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:53.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:53.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:24:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] boot 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: osdmap e50: 8 total, 8 up, 8 in 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: osdmap e51: 8 total, 8 up, 8 in 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:53.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:53.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:24:53.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: osd.7 [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] boot 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: osdmap e50: 8 total, 8 up, 8 in 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: osdmap e51: 8 total, 8 up, 8 in 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:24:53.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:24:54.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:53 smithi161 ceph-mon[129184]: pgmap v155: 1 pgs: 1 unknown; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:54.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:53 smithi161 ceph-mon[129184]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:54.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:53 smithi161 ceph-mon[129184]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T19:24:54.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:53 smithi133 ceph-mon[128894]: pgmap v155: 1 pgs: 1 unknown; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:53 smithi133 ceph-mon[128894]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:53 smithi133 ceph-mon[128894]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T19:24:54.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:53 smithi082 ceph-mon[127461]: pgmap v155: 1 pgs: 1 unknown; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-14T19:24:54.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:53 smithi082 ceph-mon[127461]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:24:54.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:53 smithi082 ceph-mon[127461]: osdmap e52: 8 total, 8 up, 8 in 2023-07-14T19:24:55.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T19:24:55.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:24:55.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/4126304854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]: dispatch 2023-07-14T19:24:55.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]: dispatch 2023-07-14T19:24:55.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]': finished 2023-07-14T19:24:55.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: osdmap e54: 9 total, 8 up, 9 in 2023-07-14T19:24:55.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:24:55.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T19:24:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:24:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/4126304854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]: dispatch 2023-07-14T19:24:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]: dispatch 2023-07-14T19:24:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]': finished 2023-07-14T19:24:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: osdmap e54: 9 total, 8 up, 9 in 2023-07-14T19:24:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:55 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:24:55.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: osdmap e53: 8 total, 8 up, 8 in 2023-07-14T19:24:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:24:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/4126304854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]: dispatch 2023-07-14T19:24:55.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]: dispatch 2023-07-14T19:24:55.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16b227e3-291b-4c62-943f-9262f1114940"}]': finished 2023-07-14T19:24:55.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: osdmap e54: 9 total, 8 up, 9 in 2023-07-14T19:24:55.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:55 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:24:56.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:56 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/3444775775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:56.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:56 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/3444775775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:56 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/3444775775' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:24:57.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:57 smithi133 ceph-mon[128894]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:24:57.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:57 smithi082 ceph-mon[127461]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:24:57.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:57 smithi161 ceph-mon[129184]: pgmap v160: 1 pgs: 1 unknown; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:00.051 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:24:59 smithi161 ceph-mon[129184]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:24:59 smithi133 ceph-mon[128894]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:24:59 smithi082 ceph-mon[127461]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:02.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:01 smithi133 ceph-mon[128894]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:02.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:01 smithi082 ceph-mon[127461]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:02.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:01 smithi161 ceph-mon[129184]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T19:25:03.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:03.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:02 smithi133 ceph-mon[128894]: Deploying daemon osd.8 on smithi161 2023-07-14T19:25:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T19:25:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:03.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:02 smithi082 ceph-mon[127461]: Deploying daemon osd.8 on smithi161 2023-07-14T19:25:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-14T19:25:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:02 smithi161 ceph-mon[129184]: Deploying daemon osd.8 on smithi161 2023-07-14T19:25:03.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:03 smithi161 ceph-mon[129184]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:04.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:03 smithi133 ceph-mon[128894]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:04.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:03 smithi082 ceph-mon[127461]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:05.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:05 smithi161 ceph-mon[129184]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:05 smithi133 ceph-mon[128894]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:06.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:05 smithi082 ceph-mon[127461]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:06.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:06.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:06.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:06 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:07.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:06 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:07.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:06 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:08.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:07 smithi133 ceph-mon[128894]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:08.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:07 smithi082 ceph-mon[127461]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:08.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:07 smithi161 ceph-mon[129184]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:09.232 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.230+0000 7fc703fff700 1 -- 172.21.15.161:0/2125171526 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc71c060a80 con 0x7fc7040a1150 2023-07-14T19:25:09.232 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2023-07-14T19:25:09.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc7040a1150 msgr2=0x7fc7040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:09.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc7040a1150 0x7fc7040a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ff3f0 tx=0x7fc710009310 comp rx=0 tx=0).stop 2023-07-14T19:25:09.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 msgr2=0x7fc71c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:09.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c19ce00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc714009f60 tx=0x7fc714004d90 comp rx=0 tx=0).stop 2023-07-14T19:25:09.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 shutdown_connections 2023-07-14T19:25:09.235 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc71c0fff20 0x7fc71c19ce00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:09.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71c0fe390 0x7fc71c10b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:09.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc7040a1150 0x7fc7040a3600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:09.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 --2- 172.21.15.161:0/2125171526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc71c0ff590 0x7fc71c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:09.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 >> 172.21.15.161:0/2125171526 conn(0x7fc71c0f99c0 msgr2=0x7fc71c0fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:25:09.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 shutdown_connections 2023-07-14T19:25:09.236 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:09.233+0000 7fc72142e700 1 -- 172.21.15.161:0/2125171526 wait complete. 2023-07-14T19:25:09.787 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.8.service 2023-07-14T19:25:09.789 INFO:tasks.cephadm:Deploying osd.9 on smithi082 with /dev/vg_nvme/lv_1... 2023-07-14T19:25:09.790 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T19:25:09.818 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:09 smithi161 ceph-mon[129184]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:09.818 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:09.818 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:09.818 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:09.818 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:09.824 INFO:journalctl@ceph.osd.8.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:25:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:09 smithi133 ceph-mon[128894]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:09 smithi082 ceph-mon[127461]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:10.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:10.357 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:25:10.660 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:25:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:25:10.502+0000 7f17bd195640 -1 osd.8 0 log_to_monitors true 2023-07-14T19:25:10.852 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:10 smithi082 ceph-mon[127461]: from='osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T19:25:10.852 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:10 smithi082 ceph-mon[127461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T19:25:10.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:10 smithi161 ceph-mon[129184]: from='osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T19:25:10.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:10 smithi161 ceph-mon[129184]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T19:25:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:10 smithi133 ceph-mon[128894]: from='osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T19:25:11.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:10 smithi133 ceph-mon[128894]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-14T19:25:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T19:25:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T19:25:11.832 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-07-14T19:25:11.832 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-07-14T19:25:11.832 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114802 s, 913 MB/s 2023-07-14T19:25:11.833 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-07-14T19:25:12.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:11 smithi082 ceph-mon[127461]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:12.134 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:11 smithi082 ceph-mon[127461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T19:25:12.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:11 smithi082 ceph-mon[127461]: osdmap e55: 9 total, 8 up, 9 in 2023-07-14T19:25:12.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:12.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:11 smithi082 ceph-mon[127461]: from='osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:25:12.135 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:11 smithi082 ceph-mon[127461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:25:12.146 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:25:11.775+0000 7f17b280d700 -1 osd.8 0 waiting for initial osdmap 2023-07-14T19:25:12.146 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:25:11.784+0000 7f17ac5ff700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:25:12.146 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-mon[129184]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:12.146 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-mon[129184]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T19:25:12.146 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-mon[129184]: osdmap e55: 9 total, 8 up, 9 in 2023-07-14T19:25:12.147 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:12.147 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-mon[129184]: from='osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:25:12.147 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:11 smithi161 ceph-mon[129184]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:25:12.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:11 smithi133 ceph-mon[128894]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:11 smithi133 ceph-mon[128894]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-14T19:25:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:11 smithi133 ceph-mon[128894]: osdmap e55: 9 total, 8 up, 9 in 2023-07-14T19:25:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:11 smithi133 ceph-mon[128894]: from='osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:25:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:11 smithi133 ceph-mon[128894]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:25:12.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2023-07-14T19:25:12.769 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:25:13.030 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:25:13.030 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T19:25:13.030 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.030 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:13.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:13.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:13.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:13.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:13.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.033 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] boot 2023-07-14T19:25:13.033 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: osdmap e57: 9 total, 9 up, 9 in 2023-07-14T19:25:13.033 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:12 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:25:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T19:25:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:13.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:13.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:13.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] boot 2023-07-14T19:25:13.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: osdmap e57: 9 total, 9 up, 9 in 2023-07-14T19:25:13.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:12 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:25:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: osdmap e56: 9 total, 8 up, 9 in 2023-07-14T19:25:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:13.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: osd.8 [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] boot 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: osdmap e57: 9 total, 9 up, 9 in 2023-07-14T19:25:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:12 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:25:13.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 -- 172.21.15.82:0/2104347628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 msgr2=0x7f56180fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:13.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 --2- 172.21.15.82:0/2104347628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f56180fefc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f560c009990 tx=0x7f560c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:25:13.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 -- 172.21.15.82:0/2104347628 shutdown_connections 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 --2- 172.21.15.82:0/2104347628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5618100740 0x7f5618107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 --2- 172.21.15.82:0/2104347628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f56180fefc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 --2- 172.21.15.82:0/2104347628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56180ffdb0 0x7f5618100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 -- 172.21.15.82:0/2104347628 >> 172.21.15.82:0/2104347628 conn(0x7f56180fa1e0 msgr2=0x7f56180fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 -- 172.21.15.82:0/2104347628 shutdown_connections 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.939+0000 7f561cb72700 1 -- 172.21.15.82:0/2104347628 wait complete. 2023-07-14T19:25:13.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 Processor -- start 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 -- start start 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f5618068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56180ffdb0 0x7f561819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5618100740 0x7f561819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819d540 con 0x7f56180febb0 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819d6b0 con 0x7f5618100740 2023-07-14T19:25:13.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561cb72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819d870 con 0x7f56180ffdb0 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f5618068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f5618068c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40184/0 (socket says 172.21.15.82:40184) 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 -- 172.21.15.82:0/4051913841 learned_addr learned my addr 172.21.15.82:0/4051913841 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f5616d9d700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5618100740 0x7f561819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f5615d9b700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56180ffdb0 0x7f561819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 -- 172.21.15.82:0/4051913841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56180ffdb0 msgr2=0x7f561819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56180ffdb0 0x7f561819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 -- 172.21.15.82:0/4051913841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5618100740 msgr2=0x7f561819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5618100740 0x7f561819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.940+0000 7f561659c700 1 -- 172.21.15.82:0/4051913841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c0095f0 con 0x7f56180febb0 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.941+0000 7f561659c700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f5618068c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f560c000c00 tx=0x7f560c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.941+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c01d070 con 0x7f56180febb0 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.941+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560c00fca0 con 0x7f56180febb0 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.941+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561819daf0 con 0x7f56180febb0 2023-07-14T19:25:13.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.941+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c017760 con 0x7f56180febb0 2023-07-14T19:25:13.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.941+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561819df00 con 0x7f56180febb0 2023-07-14T19:25:13.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.942+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56181061b0 con 0x7f56180febb0 2023-07-14T19:25:13.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.943+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f560c0178c0 con 0x7f56180febb0 2023-07-14T19:25:13.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.943+0000 7f56077fe700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f56000a1150 0x7f56000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:13.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.943+0000 7f5615d9b700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f56000a1150 0x7f56000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:13.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.943+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f560c0cd690 con 0x7f56180febb0 2023-07-14T19:25:13.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.944+0000 7f5615d9b700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f56000a1150 0x7f56000a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56180691e0 tx=0x7f5608005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:25:13.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:13.948+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f560c097800 con 0x7f56180febb0 2023-07-14T19:25:14.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:14.141+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5618060a80 con 0x7f56000a1150 2023-07-14T19:25:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:13 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:25:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:13 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:25:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:13 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi161 2023-07-14T19:25:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:13 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi161 to 5722M 2023-07-14T19:25:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:13 smithi133 ceph-mon[128894]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:14.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:13 smithi133 ceph-mon[128894]: osdmap e58: 9 total, 9 up, 9 in 2023-07-14T19:25:14.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:13 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:25:14.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:13 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:25:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:13 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi161 2023-07-14T19:25:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:13 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi161 to 5722M 2023-07-14T19:25:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:13 smithi082 ceph-mon[127461]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:14.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:13 smithi082 ceph-mon[127461]: osdmap e58: 9 total, 9 up, 9 in 2023-07-14T19:25:14.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:13 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:25:14.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:13 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:25:14.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:13 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi161 2023-07-14T19:25:14.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:13 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi161 to 5722M 2023-07-14T19:25:14.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:13 smithi161 ceph-mon[129184]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-14T19:25:14.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:13 smithi161 ceph-mon[129184]: osdmap e58: 9 total, 9 up, 9 in 2023-07-14T19:25:15.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:14 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:15.031 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:14 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:15.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:14 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:15.032 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:14 smithi082 ceph-mon[127461]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T19:25:15.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:14 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:15.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:14 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:15.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:14 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:15.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:14 smithi133 ceph-mon[128894]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T19:25:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:14 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:15.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:14 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:15.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:14 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:15.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:14 smithi161 ceph-mon[129184]: osdmap e59: 9 total, 9 up, 9 in 2023-07-14T19:25:16.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:15 smithi133 ceph-mon[128894]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:16.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:15 smithi133 ceph-mon[128894]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:16.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:15 smithi082 ceph-mon[127461]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:16.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:15 smithi082 ceph-mon[127461]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:16.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:15 smithi161 ceph-mon[129184]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:16.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:15 smithi161 ceph-mon[129184]: pgmap v174: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:17.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:16 smithi133 ceph-mon[128894]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T19:25:17.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:16 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4290185385' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "300cc59c-8947-4ec0-9352-784783f12ff2"}]: dispatch 2023-07-14T19:25:17.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:16 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4290185385' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "300cc59c-8947-4ec0-9352-784783f12ff2"}]': finished 2023-07-14T19:25:17.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:16 smithi133 ceph-mon[128894]: osdmap e61: 10 total, 9 up, 10 in 2023-07-14T19:25:17.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:16 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:17.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:16 smithi082 ceph-mon[127461]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T19:25:17.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:16 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4290185385' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "300cc59c-8947-4ec0-9352-784783f12ff2"}]: dispatch 2023-07-14T19:25:17.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:16 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4290185385' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "300cc59c-8947-4ec0-9352-784783f12ff2"}]': finished 2023-07-14T19:25:17.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:16 smithi082 ceph-mon[127461]: osdmap e61: 10 total, 9 up, 10 in 2023-07-14T19:25:17.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:17.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:16 smithi161 ceph-mon[129184]: osdmap e60: 9 total, 9 up, 9 in 2023-07-14T19:25:17.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:16 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4290185385' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "300cc59c-8947-4ec0-9352-784783f12ff2"}]: dispatch 2023-07-14T19:25:17.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:16 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4290185385' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "300cc59c-8947-4ec0-9352-784783f12ff2"}]': finished 2023-07-14T19:25:17.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:16 smithi161 ceph-mon[129184]: osdmap e61: 10 total, 9 up, 10 in 2023-07-14T19:25:17.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:16 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:18.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:17 smithi133 ceph-mon[128894]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:18.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:17 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2337145743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:25:18.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:17 smithi082 ceph-mon[127461]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:18.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:17 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2337145743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:25:18.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:17 smithi161 ceph-mon[129184]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:18.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:17 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2337145743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:25:20.156 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:19 smithi082 ceph-mon[127461]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-07-14T19:25:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:19 smithi133 ceph-mon[128894]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-07-14T19:25:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:19 smithi161 ceph-mon[129184]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-07-14T19:25:22.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:21 smithi133 ceph-mon[128894]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-14T19:25:22.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:21 smithi082 ceph-mon[127461]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-14T19:25:22.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:21 smithi161 ceph-mon[129184]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-07-14T19:25:23.818 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:23 smithi082 ceph-mon[127461]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:25:23.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T19:25:23.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:23 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:23.819 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:23 smithi082 ceph-mon[127461]: Deploying daemon osd.9 on smithi082 2023-07-14T19:25:23.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:23 smithi133 ceph-mon[128894]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:25:23.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T19:25:23.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:23 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:23.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:23 smithi133 ceph-mon[128894]: Deploying daemon osd.9 on smithi082 2023-07-14T19:25:23.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:23 smithi161 ceph-mon[129184]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-14T19:25:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-14T19:25:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:23 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:23 smithi161 ceph-mon[129184]: Deploying daemon osd.9 on smithi082 2023-07-14T19:25:26.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:25 smithi133 ceph-mon[128894]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-14T19:25:26.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:25 smithi082 ceph-mon[127461]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-14T19:25:26.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:25 smithi161 ceph-mon[129184]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-14T19:25:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:27 smithi133 ceph-mon[128894]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-14T19:25:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:28.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:28.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:28.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:27 smithi082 ceph-mon[127461]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-14T19:25:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:28.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:27 smithi161 ceph-mon[129184]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-07-14T19:25:28.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:30.181 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:29 smithi082 ceph-mon[127461]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:25:30.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:29 smithi133 ceph-mon[128894]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:25:30.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.193+0000 7f56077fe700 1 -- 172.21.15.82:0/4051913841 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5618060a80 con 0x7f56000a1150 2023-07-14T19:25:30.195 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 9 on host 'smithi082' 2023-07-14T19:25:30.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.195+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f56000a1150 msgr2=0x7f56000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.195+0000 7f561cb72700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f56000a1150 0x7f56000a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f56180691e0 tx=0x7f5608005cf0 comp rx=0 tx=0).stop 2023-07-14T19:25:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.195+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 msgr2=0x7f5618068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.195+0000 7f561cb72700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f5618068c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f560c000c00 tx=0x7f560c00f7b0 comp rx=0 tx=0).stop 2023-07-14T19:25:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.195+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 shutdown_connections 2023-07-14T19:25:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.195+0000 7f561cb72700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5618100740 0x7f561819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.196+0000 7f561cb72700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f56000a1150 0x7f56000a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.196+0000 7f561cb72700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56180febb0 0x7f5618068c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.196+0000 7f561cb72700 1 --2- 172.21.15.82:0/4051913841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56180ffdb0 0x7f561819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.196+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 >> 172.21.15.82:0/4051913841 conn(0x7f56180fa1e0 msgr2=0x7f56181032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:25:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.196+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 shutdown_connections 2023-07-14T19:25:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:25:30.196+0000 7f561cb72700 1 -- 172.21.15.82:0/4051913841 wait complete. 2023-07-14T19:25:30.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:29 smithi161 ceph-mon[129184]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-14T19:25:30.654 DEBUG:teuthology.orchestra.run.smithi082:osd.9> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.9.service 2023-07-14T19:25:30.657 INFO:tasks.cephadm:Deploying osd.10 on smithi133 with /dev/vg_nvme/lv_1... 2023-07-14T19:25:30.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T19:25:30.691 INFO:journalctl@ceph.osd.9.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:25:31.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.129 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.222 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:25:31.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:31.671 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:25:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:25:31.430+0000 7fb693b78640 -1 osd.9 0 log_to_monitors true 2023-07-14T19:25:31.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:31 smithi133 ceph-mon[128894]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:31.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:31 smithi133 ceph-mon[128894]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T19:25:31.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:31 smithi082 ceph-mon[127461]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:31.938 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:31 smithi082 ceph-mon[127461]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T19:25:32.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:31 smithi161 ceph-mon[129184]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:31 smithi161 ceph-mon[129184]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-14T19:25:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T19:25:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T19:25:32.607 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-07-14T19:25:32.607 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-07-14T19:25:32.607 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111338 s, 942 MB/s 2023-07-14T19:25:32.608 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-07-14T19:25:32.969 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2023-07-14T19:25:33.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T19:25:33.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: osdmap e62: 10 total, 9 up, 10 in 2023-07-14T19:25:33.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:33.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:25:33.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:33.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi082 2023-07-14T19:25:33.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi082 to 4292M 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:33.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:33.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:33.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:33.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:33.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.464 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:25:33.162+0000 7fb6891f0700 -1 osd.9 0 waiting for initial osdmap 2023-07-14T19:25:33.464 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:25:33.175+0000 7fb683fe4700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:25:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T19:25:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: osdmap e62: 10 total, 9 up, 10 in 2023-07-14T19:25:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:25:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:33.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi082 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi082 to 4292M 2023-07-14T19:25:33.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:33.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-14T19:25:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: osdmap e62: 10 total, 9 up, 10 in 2023-07-14T19:25:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-07-14T19:25:33.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:33.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi082 2023-07-14T19:25:33.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi082 to 4292M 2023-07-14T19:25:33.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:33.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:33.524 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:25:34.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:34 smithi133 ceph-mon[128894]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:25:34.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:34 smithi133 ceph-mon[128894]: osdmap e63: 10 total, 9 up, 10 in 2023-07-14T19:25:34.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:34 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:34.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:34 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:34.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:34 smithi082 ceph-mon[127461]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:25:34.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:34 smithi082 ceph-mon[127461]: osdmap e63: 10 total, 9 up, 10 in 2023-07-14T19:25:34.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:34 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:34.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:34 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:34.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:34 smithi161 ceph-mon[129184]: from='osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-07-14T19:25:34.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:34 smithi161 ceph-mon[129184]: osdmap e63: 10 total, 9 up, 10 in 2023-07-14T19:25:34.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:34 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:34.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:34 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 -- 172.21.15.133:0/3158174890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101059e0 msgr2=0x7ff410107de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 --2- 172.21.15.133:0/3158174890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101059e0 0x7ff410107de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff40800b0a0 tx=0x7ff40800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:25:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 -- 172.21.15.133:0/3158174890 shutdown_connections 2023-07-14T19:25:34.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 --2- 172.21.15.133:0/3158174890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101059e0 0x7ff410107de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:34.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 --2- 172.21.15.133:0/3158174890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101007a0 0x7ff410102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:34.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 --2- 172.21.15.133:0/3158174890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101030c0 0x7ff4101054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:34.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 -- 172.21.15.133:0/3158174890 >> 172.21.15.133:0/3158174890 conn(0x7ff4100fa1e0 msgr2=0x7ff4100fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:25:34.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 -- 172.21.15.133:0/3158174890 shutdown_connections 2023-07-14T19:25:34.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.723+0000 7ff415645700 1 -- 172.21.15.133:0/3158174890 wait complete. 2023-07-14T19:25:34.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 Processor -- start 2023-07-14T19:25:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 -- start start 2023-07-14T19:25:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 0x7ff410195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101030c0 0x7ff410195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101059e0 0x7ff41019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41019d5a0 con 0x7ff4101030c0 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41019d6e0 con 0x7ff4101007a0 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff415645700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41019d870 con 0x7ff4101059e0 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff40effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 0x7ff410195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff40effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 0x7ff410195a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39152/0 (socket says 172.21.15.133:39152) 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.724+0000 7ff40effd700 1 -- 172.21.15.133:0/598530463 learned_addr learned my addr 172.21.15.133:0/598530463 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40e7fc700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101030c0 0x7ff410195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:34.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40f7fe700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101059e0 0x7ff41019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40effd700 1 -- 172.21.15.133:0/598530463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101059e0 msgr2=0x7ff41019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40effd700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101059e0 0x7ff41019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40effd700 1 -- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101030c0 msgr2=0x7ff410195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40effd700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101030c0 0x7ff410195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40effd700 1 -- 172.21.15.133:0/598530463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408009d00 con 0x7ff4101007a0 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40f7fe700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101059e0 0x7ff41019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:25:34.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40e7fc700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101030c0 0x7ff410195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff40effd700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 0x7ff410195a10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c9b0 tx=0x7ff40400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404007b00 con 0x7ff4101007a0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff404007c60 con 0x7ff4101007a0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41019db50 con 0x7ff4101007a0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404018640 con 0x7ff4101007a0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.725+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41019e050 con 0x7ff4101007a0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.726+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4100fbd50 con 0x7ff4101007a0 2023-07-14T19:25:34.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.727+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff404018860 con 0x7ff4101007a0 2023-07-14T19:25:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.728+0000 7ff3f7fff700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff3f80a1150 0x7ff3f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.728+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7ff4040cc440 con 0x7ff4101007a0 2023-07-14T19:25:34.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.728+0000 7ff40e7fc700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff3f80a1150 0x7ff3f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.729+0000 7ff40e7fc700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff3f80a1150 0x7ff3f80a3600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff400005950 tx=0x7ff4000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:25:34.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.731+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff404096420 con 0x7ff4101007a0 2023-07-14T19:25:34.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:34.924+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff410060a80 con 0x7ff3f80a1150 2023-07-14T19:25:35.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:25:35.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:25:35.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] boot 2023-07-14T19:25:35.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: osdmap e64: 10 total, 10 up, 10 in 2023-07-14T19:25:35.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:35.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:35.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:35.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:35.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:35.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:35.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:25:35.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:25:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] boot 2023-07-14T19:25:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: osdmap e64: 10 total, 10 up, 10 in 2023-07-14T19:25:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:35.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:35.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:35.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:35.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:25:35.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:25:35.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: osd.9 [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] boot 2023-07-14T19:25:35.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: osdmap e64: 10 total, 10 up, 10 in 2023-07-14T19:25:35.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:25:35.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-14T19:25:35.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:35.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:35.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:35.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:36.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:36 smithi133 ceph-mon[128894]: osdmap e65: 10 total, 10 up, 10 in 2023-07-14T19:25:36.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:36 smithi082 ceph-mon[127461]: osdmap e65: 10 total, 10 up, 10 in 2023-07-14T19:25:36.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:36 smithi161 ceph-mon[129184]: osdmap e65: 10 total, 10 up, 10 in 2023-07-14T19:25:37.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:37 smithi082 ceph-mon[127461]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:37.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:37 smithi161 ceph-mon[129184]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:37.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:37 smithi133 ceph-mon[128894]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:38.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:38 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/1155604593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]: dispatch 2023-07-14T19:25:38.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:38 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]: dispatch 2023-07-14T19:25:38.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:38 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]': finished 2023-07-14T19:25:38.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:38 smithi082 ceph-mon[127461]: osdmap e66: 11 total, 10 up, 11 in 2023-07-14T19:25:38.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:38.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:38 smithi082 ceph-mon[127461]: from='client.? 172.21.15.133:0/3149117472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:25:38.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:38 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/1155604593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]: dispatch 2023-07-14T19:25:38.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:38 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]: dispatch 2023-07-14T19:25:38.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:38 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]': finished 2023-07-14T19:25:38.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:38 smithi161 ceph-mon[129184]: osdmap e66: 11 total, 10 up, 11 in 2023-07-14T19:25:38.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:38.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:38 smithi161 ceph-mon[129184]: from='client.? 172.21.15.133:0/3149117472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:25:38.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:38 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/1155604593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]: dispatch 2023-07-14T19:25:38.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:38 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]: dispatch 2023-07-14T19:25:38.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:38 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13ec79ce-2a2b-4930-83e8-98530ae6fe96"}]': finished 2023-07-14T19:25:38.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:38 smithi133 ceph-mon[128894]: osdmap e66: 11 total, 10 up, 11 in 2023-07-14T19:25:38.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:38.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:38 smithi133 ceph-mon[128894]: from='client.? 172.21.15.133:0/3149117472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:25:39.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:39 smithi082 ceph-mon[127461]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:39.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:39 smithi161 ceph-mon[129184]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:39.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:39 smithi133 ceph-mon[128894]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:42.172 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:41 smithi133 ceph-mon[128894]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:42.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:41 smithi082 ceph-mon[127461]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:42.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:41 smithi161 ceph-mon[129184]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:43.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:43 smithi133 ceph-mon[128894]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:43.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:43 smithi082 ceph-mon[127461]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:43.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:43 smithi161 ceph-mon[129184]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:44.780 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:44 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T19:25:44.781 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:44 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:44.781 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:44 smithi133 ceph-mon[128894]: Deploying daemon osd.10 on smithi133 2023-07-14T19:25:44.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:44 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T19:25:44.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:44 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:44.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:44 smithi082 ceph-mon[127461]: Deploying daemon osd.10 on smithi133 2023-07-14T19:25:44.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:44 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-14T19:25:44.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:44 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:44.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:44 smithi161 ceph-mon[129184]: Deploying daemon osd.10 on smithi133 2023-07-14T19:25:45.611 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:45 smithi133 ceph-mon[128894]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:45.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:45 smithi082 ceph-mon[127461]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:45.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:45 smithi161 ceph-mon[129184]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:48.104 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:47 smithi133 ceph-mon[128894]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:48.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:47 smithi082 ceph-mon[127461]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:48.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:47 smithi161 ceph-mon[129184]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:49.330 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:49.330 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:49.331 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:49.331 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:49 smithi133 ceph-mon[128894]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:49.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:49.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:49.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:49.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:49 smithi161 ceph-mon[129184]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:49.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:49.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:49.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:49.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:49 smithi082 ceph-mon[127461]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:50.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.883+0000 7ff3f7fff700 1 -- 172.21.15.133:0/598530463 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff410060a80 con 0x7ff3f80a1150 2023-07-14T19:25:50.885 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 10 on host 'smithi133' 2023-07-14T19:25:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff3f80a1150 msgr2=0x7ff3f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff3f80a1150 0x7ff3f80a3600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff400005950 tx=0x7ff4000058e0 comp rx=0 tx=0).stop 2023-07-14T19:25:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 msgr2=0x7ff410195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 0x7ff410195a10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff40400c9b0 tx=0x7ff40400cd70 comp rx=0 tx=0).stop 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 shutdown_connections 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4101059e0 0x7ff41019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4101007a0 0x7ff410195a10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff3f80a1150 0x7ff3f80a3600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 --2- 172.21.15.133:0/598530463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4101030c0 0x7ff410195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.885+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 >> 172.21.15.133:0/598530463 conn(0x7ff4100fa1e0 msgr2=0x7ff410101400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:25:50.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.886+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 shutdown_connections 2023-07-14T19:25:50.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-07-14T19:25:50.886+0000 7ff415645700 1 -- 172.21.15.133:0/598530463 wait complete. 2023-07-14T19:25:51.322 DEBUG:teuthology.orchestra.run.smithi133:osd.10> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.10.service 2023-07-14T19:25:51.324 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2023-07-14T19:25:51.324 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-14T19:25:51.354 INFO:journalctl@ceph.osd.10.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:25:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:25:52.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:51 smithi161 ceph-mon[129184]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:52.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:51 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:51 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:51 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:51 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.177 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:51 smithi133 ceph-mon[128894]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:52.177 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:51 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.178 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:51 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.178 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:51 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.178 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:51 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:51 smithi082 ceph-mon[127461]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:52.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:51 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:51 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:51 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:51 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:52.434 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:25:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:25:52.235+0000 7f720f8ec640 -1 osd.10 0 log_to_monitors true 2023-07-14T19:25:52.984 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:52 smithi161 ceph-mon[129184]: from='osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T19:25:52.985 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:52 smithi161 ceph-mon[129184]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T19:25:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:52 smithi133 ceph-mon[128894]: from='osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T19:25:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:52 smithi133 ceph-mon[128894]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T19:25:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:52 smithi082 ceph-mon[127461]: from='osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T19:25:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:52 smithi082 ceph-mon[127461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-14T19:25:53.489 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-14T19:25:53.489 INFO:teuthology.orchestra.run.smithi161.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-14T19:25:53.489 INFO:teuthology.orchestra.run.smithi161.stderr: stderr: 10+0 records in 2023-07-14T19:25:53.489 INFO:teuthology.orchestra.run.smithi161.stderr:10+0 records out 2023-07-14T19:25:53.490 INFO:teuthology.orchestra.run.smithi161.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111318 s, 942 MB/s 2023-07-14T19:25:53.490 INFO:teuthology.orchestra.run.smithi161.stderr:--> Zapping successful for: 2023-07-14T19:25:53.940 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2023-07-14T19:25:54.183 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:25:53.856+0000 7f7204f64700 -1 osd.10 0 waiting for initial osdmap 2023-07-14T19:25:54.183 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:25:53.869+0000 7f71fed56700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:25:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-mon[128894]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-mon[128894]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T19:25:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-mon[128894]: osdmap e67: 11 total, 10 up, 11 in 2023-07-14T19:25:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:54.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-mon[128894]: from='osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:25:54.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:53 smithi133 ceph-mon[128894]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:25:54.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:53 smithi082 ceph-mon[127461]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:54.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:53 smithi082 ceph-mon[127461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T19:25:54.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:53 smithi082 ceph-mon[127461]: osdmap e67: 11 total, 10 up, 11 in 2023-07-14T19:25:54.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:53 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:54.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:53 smithi082 ceph-mon[127461]: from='osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:25:54.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:53 smithi082 ceph-mon[127461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:25:54.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:53 smithi161 ceph-mon[129184]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:54.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:53 smithi161 ceph-mon[129184]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-14T19:25:54.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:53 smithi161 ceph-mon[129184]: osdmap e67: 11 total, 10 up, 11 in 2023-07-14T19:25:54.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:53 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:54.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:53 smithi161 ceph-mon[129184]: from='osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:25:54.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:53 smithi161 ceph-mon[129184]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-07-14T19:25:54.496 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:25:54.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi133 2023-07-14T19:25:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:25:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: osdmap e68: 11 total, 10 up, 11 in 2023-07-14T19:25:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi133 to 4292M 2023-07-14T19:25:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:54.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi133 2023-07-14T19:25:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:25:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: osdmap e68: 11 total, 10 up, 11 in 2023-07-14T19:25:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi133 to 4292M 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:55.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:55.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:55.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:55.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:55.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi133 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: osdmap e68: 11 total, 10 up, 11 in 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi133 to 4292M 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:55.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:25:55.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:55.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:25:55.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:25:55.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 -- 172.21.15.161:0/464825309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 msgr2=0x7f8d98100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:55.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/464825309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98100200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009990 tx=0x7f8d84009ca0 comp rx=0 tx=0).stop 2023-07-14T19:25:55.715 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 -- 172.21.15.161:0/464825309 shutdown_connections 2023-07-14T19:25:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/464825309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d98100740 0x7f8d98107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/464825309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d980febb0 0x7f8d980fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/464825309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98100200 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 -- 172.21.15.161:0/464825309 >> 172.21.15.161:0/464825309 conn(0x7f8d980fa1e0 msgr2=0x7f8d980fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:25:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 -- 172.21.15.161:0/464825309 shutdown_connections 2023-07-14T19:25:55.716 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 -- 172.21.15.161:0/464825309 wait complete. 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.712+0000 7f8d9ef35700 1 Processor -- start 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 -- start start 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d980febb0 0x7f8d981959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d98100740 0x7f8d9819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819d550 con 0x7f8d98100740 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819d690 con 0x7f8d980febb0 2023-07-14T19:25:55.717 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819d820 con 0x7f8d980ffdb0 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60460/0 (socket says 172.21.15.161:60460) 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 -- 172.21.15.161:0/3373488469 learned_addr learned my addr 172.21.15.161:0/3373488469 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ccd1700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d980febb0 0x7f8d981959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9d4d2700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d98100740 0x7f8d9819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 -- 172.21.15.161:0/3373488469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d980febb0 msgr2=0x7f8d981959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:55.718 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d980febb0 0x7f8d981959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 -- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d98100740 msgr2=0x7f8d9819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d98100740 0x7f8d9819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 -- 172.21.15.161:0/3373488469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840095f0 con 0x7f8d980ffdb0 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d97fff700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98195f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d84006010 tx=0x7f8d8400f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8401d070 con 0x7f8d980ffdb0 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8400fd60 con 0x7f8d980ffdb0 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9819daa0 con 0x7f8d980ffdb0 2023-07-14T19:25:55.719 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.713+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84017650 con 0x7f8d980ffdb0 2023-07-14T19:25:55.720 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.714+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9819deb0 con 0x7f8d980ffdb0 2023-07-14T19:25:55.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.715+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9810b520 con 0x7f8d980ffdb0 2023-07-14T19:25:55.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.716+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8d840177b0 con 0x7f8d980ffdb0 2023-07-14T19:25:55.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.716+0000 7f8d95ffb700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d880a1150 0x7f8d880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:25:55.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.716+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f8d840cd850 con 0x7f8d980ffdb0 2023-07-14T19:25:55.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.716+0000 7f8d9ccd1700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d880a1150 0x7f8d880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:25:55.721 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.717+0000 7f8d9ccd1700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d880a1150 0x7f8d880a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005950 tx=0x7f8d8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:25:55.722 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.720+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8d8405c780 con 0x7f8d980ffdb0 2023-07-14T19:25:55.916 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:25:55.915+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d98060a80 con 0x7f8d880a1150 2023-07-14T19:25:56.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:25:56.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:25:56.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:56.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:56.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] boot 2023-07-14T19:25:56.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: osdmap e69: 11 total, 11 up, 11 in 2023-07-14T19:25:56.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:56.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:25:56.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:25:56.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:56.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:56.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] boot 2023-07-14T19:25:56.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: osdmap e69: 11 total, 11 up, 11 in 2023-07-14T19:25:56.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:55 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:56.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:25:56.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:25:56.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-14T19:25:56.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:56.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: osd.10 [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] boot 2023-07-14T19:25:56.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: osdmap e69: 11 total, 11 up, 11 in 2023-07-14T19:25:56.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:55 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:25:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:56 smithi161 ceph-mon[129184]: osdmap e70: 11 total, 11 up, 11 in 2023-07-14T19:25:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:56 smithi161 ceph-mon[129184]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:56.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:56.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:56.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:56 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:56 smithi133 ceph-mon[128894]: osdmap e70: 11 total, 11 up, 11 in 2023-07-14T19:25:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:56 smithi133 ceph-mon[128894]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:56 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:57.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:56 smithi082 ceph-mon[127461]: osdmap e70: 11 total, 11 up, 11 in 2023-07-14T19:25:57.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:56 smithi082 ceph-mon[127461]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:25:57.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-14T19:25:57.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-14T19:25:57.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:56 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:25:58.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:57 smithi133 ceph-mon[128894]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:25:58.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:57 smithi133 ceph-mon[128894]: osdmap e71: 11 total, 11 up, 11 in 2023-07-14T19:25:58.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:57 smithi133 ceph-mon[128894]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T19:25:58.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:57 smithi082 ceph-mon[127461]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:25:58.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:57 smithi082 ceph-mon[127461]: osdmap e71: 11 total, 11 up, 11 in 2023-07-14T19:25:58.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:57 smithi082 ceph-mon[127461]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T19:25:58.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:57 smithi161 ceph-mon[129184]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:25:58.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:57 smithi161 ceph-mon[129184]: osdmap e71: 11 total, 11 up, 11 in 2023-07-14T19:25:58.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:57 smithi161 ceph-mon[129184]: osdmap e72: 11 total, 11 up, 11 in 2023-07-14T19:25:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:58 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/1242252818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]: dispatch 2023-07-14T19:25:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:58 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]: dispatch 2023-07-14T19:25:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:58 smithi133 ceph-mon[128894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]': finished 2023-07-14T19:25:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:58 smithi133 ceph-mon[128894]: osdmap e73: 12 total, 11 up, 12 in 2023-07-14T19:25:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:58 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:25:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:58 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/1242252818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]: dispatch 2023-07-14T19:25:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:58 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]: dispatch 2023-07-14T19:25:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:58 smithi082 ceph-mon[127461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]': finished 2023-07-14T19:25:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:58 smithi082 ceph-mon[127461]: osdmap e73: 12 total, 11 up, 12 in 2023-07-14T19:25:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:58 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:25:59.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:58 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/1242252818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]: dispatch 2023-07-14T19:25:59.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:58 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]: dispatch 2023-07-14T19:25:59.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:58 smithi161 ceph-mon[129184]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9803dc37-4330-4d09-b2db-38f8fac795c8"}]': finished 2023-07-14T19:25:59.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:58 smithi161 ceph-mon[129184]: osdmap e73: 12 total, 11 up, 12 in 2023-07-14T19:25:59.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:58 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:59 smithi133 ceph-mon[128894]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 150 KiB/s, 0 objects/s recovering 2023-07-14T19:26:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:25:59 smithi133 ceph-mon[128894]: from='client.? 172.21.15.161:0/3467171190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:26:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:59 smithi082 ceph-mon[127461]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 150 KiB/s, 0 objects/s recovering 2023-07-14T19:26:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:25:59 smithi082 ceph-mon[127461]: from='client.? 172.21.15.161:0/3467171190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:26:00.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:59 smithi161 ceph-mon[129184]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 150 KiB/s, 0 objects/s recovering 2023-07-14T19:26:00.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:25:59 smithi161 ceph-mon[129184]: from='client.? 172.21.15.161:0/3467171190' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-14T19:26:02.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:01 smithi133 ceph-mon[128894]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2023-07-14T19:26:02.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:01 smithi082 ceph-mon[127461]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2023-07-14T19:26:02.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:01 smithi161 ceph-mon[129184]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2023-07-14T19:26:03.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:03 smithi133 ceph-mon[128894]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T19:26:03.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:03 smithi082 ceph-mon[127461]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T19:26:03.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:03 smithi161 ceph-mon[129184]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-14T19:26:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:05 smithi133 ceph-mon[128894]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-14T19:26:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T19:26:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:05 smithi133 ceph-mon[128894]: Deploying daemon osd.11 on smithi161 2023-07-14T19:26:06.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:05 smithi082 ceph-mon[127461]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-14T19:26:06.228 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T19:26:06.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:06.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:05 smithi082 ceph-mon[127461]: Deploying daemon osd.11 on smithi161 2023-07-14T19:26:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:05 smithi161 ceph-mon[129184]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-07-14T19:26:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-14T19:26:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:05 smithi161 ceph-mon[129184]: Deploying daemon osd.11 on smithi161 2023-07-14T19:26:07.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:07 smithi161 ceph-mon[129184]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-14T19:26:08.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:07 smithi133 ceph-mon[128894]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-14T19:26:08.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:07 smithi082 ceph-mon[127461]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-07-14T19:26:09.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:09 smithi161 ceph-mon[129184]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:09.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:09.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:09.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:09 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:10.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:09 smithi133 ceph-mon[128894]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:09 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:10.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:09 smithi082 ceph-mon[127461]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:10.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:10.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:10.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:09 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:11.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:11 smithi161 ceph-mon[129184]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:11.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:11.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:11 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.077 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.076+0000 7f8d95ffb700 1 -- 172.21.15.161:0/3373488469 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8d98060a80 con 0x7f8d880a1150 2023-07-14T19:26:12.077 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2023-07-14T19:26:12.079 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d880a1150 msgr2=0x7f8d880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d880a1150 0x7f8d880a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005950 tx=0x7f8d8c0058e0 comp rx=0 tx=0).stop 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 msgr2=0x7f8d98195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98195f00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8d84006010 tx=0x7f8d8400f510 comp rx=0 tx=0).stop 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 shutdown_connections 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8d880a1150 0x7f8d880a3600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d98100740 0x7f8d9819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:12.080 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d980febb0 0x7f8d981959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 --2- 172.21.15.161:0/3373488469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d980ffdb0 0x7f8d98195f00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 >> 172.21.15.161:0/3373488469 conn(0x7f8d980fa1e0 msgr2=0x7f8d980fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 shutdown_connections 2023-07-14T19:26:12.081 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:12.079+0000 7f8d9ef35700 1 -- 172.21.15.161:0/3373488469 wait complete. 2023-07-14T19:26:12.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:11 smithi133 ceph-mon[128894]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:12.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:11 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:11 smithi082 ceph-mon[127461]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:12.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:12.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:12.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:11 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:12.490 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.11.service 2023-07-14T19:26:12.492 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-14T19:26:12.492 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T19:26:12.523 INFO:journalctl@ceph.osd.11.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:26:12.959 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:26:13.342 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:13.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:13.343 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:13 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:13.375 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:13.376 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:13.376 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:13 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:13.376 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:26:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:26:13.192+0000 7f298952b640 -1 osd.11 0 log_to_monitors true 2023-07-14T19:26:13.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:13.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:13.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:13 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 -- 172.21.15.82:0/1015072801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 msgr2=0x7f57c41000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/1015072801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 0x7f57c41000d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009990 tx=0x7f57b8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 -- 172.21.15.82:0/1015072801 shutdown_connections 2023-07-14T19:26:14.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/1015072801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c4101850 0x7f57c4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/1015072801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 0x7f57c41000d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/1015072801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4100ec0 0x7f57c4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 -- 172.21.15.82:0/1015072801 >> 172.21.15.82:0/1015072801 conn(0x7f57c40fb2d0 msgr2=0x7f57c40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.126+0000 7f57c8bbb700 1 -- 172.21.15.82:0/1015072801 shutdown_connections 2023-07-14T19:26:14.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.127+0000 7f57c8bbb700 1 -- 172.21.15.82:0/1015072801 wait complete. 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 Processor -- start 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 -- start start 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 0x7f57c40743e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c4100ec0 0x7f57c4072a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 0x7f57c4072f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40736f0 con 0x7f57c40ffcc0 2023-07-14T19:26:14.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4073860 con 0x7f57c4100ec0 2023-07-14T19:26:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c8bbb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c419fa80 con 0x7f57c4101850 2023-07-14T19:26:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 0x7f57c4072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 0x7f57c4072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:49606/0 (socket says 172.21.15.82:49606) 2023-07-14T19:26:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c4100ec0 0x7f57c4072a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 -- 172.21.15.82:0/3402051939 learned_addr learned my addr 172.21.15.82:0/3402051939 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c259c700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 0x7f57c40743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 -- 172.21.15.82:0/3402051939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c4100ec0 msgr2=0x7f57c4072a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c4100ec0 0x7f57c4072a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 -- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 msgr2=0x7f57c40743e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.128+0000 7f57c2d9d700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 0x7f57c40743e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57c2d9d700 1 -- 172.21.15.82:0/3402051939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80095f0 con 0x7f57c4101850 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57c2d9d700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 0x7f57c4072f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00ada0 tx=0x7f57bc00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00f070 con 0x7f57c4101850 2023-07-14T19:26:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c419fd60 con 0x7f57c4101850 2023-07-14T19:26:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57bc007b30 con 0x7f57c4101850 2023-07-14T19:26:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc005280 con 0x7f57c4101850 2023-07-14T19:26:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.129+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c41a01b0 con 0x7f57c4101850 2023-07-14T19:26:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.130+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c4107300 con 0x7f57c4101850 2023-07-14T19:26:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.131+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f57bc00a090 con 0x7f57c4101850 2023-07-14T19:26:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.131+0000 7f57b37fe700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f57ac0a1150 0x7f57ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.132+0000 7f57c259c700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f57ac0a1150 0x7f57ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.132+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f57bc0cb7f0 con 0x7f57c4101850 2023-07-14T19:26:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.132+0000 7f57c259c700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f57ac0a1150 0x7f57ac0a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b5c0 tx=0x7f57b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.135+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f57bc0955a0 con 0x7f57c4101850 2023-07-14T19:26:14.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:14 smithi082 ceph-mon[127461]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:14.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:14 smithi082 ceph-mon[127461]: from='osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T19:26:14.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:14 smithi082 ceph-mon[127461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T19:26:14.248 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:14 smithi161 ceph-mon[129184]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:14.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:14 smithi161 ceph-mon[129184]: from='osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T19:26:14.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:14 smithi161 ceph-mon[129184]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T19:26:14.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.323+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f57c410b540 con 0x7f57c4101850 2023-07-14T19:26:14.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.324+0000 7f57b37fe700 1 -- 172.21.15.82:0/3402051939 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7f57bc098bc0 con 0x7f57c4101850 2023-07-14T19:26:14.325 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f57ac0a1150 msgr2=0x7f57ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f57ac0a1150 0x7f57ac0a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b5c0 tx=0x7f57b8005fd0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 msgr2=0x7f57c4072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 0x7f57c4072f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00ada0 tx=0x7f57bc00c330 comp rx=0 tx=0).stop 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 shutdown_connections 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4101850 0x7f57c4072f70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f57ac0a1150 0x7f57ac0a3600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57c40ffcc0 0x7f57c40743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 --2- 172.21.15.82:0/3402051939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c4100ec0 0x7f57c4072a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:14.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 >> 172.21.15.82:0/3402051939 conn(0x7f57c40fb2d0 msgr2=0x7f57c4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 shutdown_connections 2023-07-14T19:26:14.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:14.326+0000 7f57c8bbb700 1 -- 172.21.15.82:0/3402051939 wait complete. 2023-07-14T19:26:14.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:14 smithi133 ceph-mon[128894]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:14.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:14 smithi133 ceph-mon[128894]: from='osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T19:26:14.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:14 smithi133 ceph-mon[128894]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-14T19:26:14.714 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1689362754,"num_in_osds":12,"osd_in_since":1689362758,"num_remapped_pgs":0} 2023-07-14T19:26:15.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: osdmap e74: 12 total, 11 up, 12 in 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3402051939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:15.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:15.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:15 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T19:26:15.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: osdmap e74: 12 total, 11 up, 12 in 2023-07-14T19:26:15.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3402051939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:15.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:15.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:15 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-14T19:26:15.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: osdmap e74: 12 total, 11 up, 12 in 2023-07-14T19:26:15.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3402051939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:26:15.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:15.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:15.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:15.495 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:26:15.070+0000 7f297eba3700 -1 osd.11 0 waiting for initial osdmap 2023-07-14T19:26:15.495 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:26:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:26:15.082+0000 7f2976991700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-14T19:26:15.715 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd stat -f json 2023-07-14T19:26:16.276 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:26:16.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:16.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: Detected new or changed devices on smithi161 2023-07-14T19:26:16.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi161 to 4292M 2023-07-14T19:26:16.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:26:16.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: osdmap e75: 12 total, 11 up, 12 in 2023-07-14T19:26:16.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:16.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:16 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:16.452 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:16.452 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: Detected new or changed devices on smithi161 2023-07-14T19:26:16.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi161 to 4292M 2023-07-14T19:26:16.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:26:16.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: osdmap e75: 12 total, 11 up, 12 in 2023-07-14T19:26:16.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:16.453 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:16 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: Detected new or changed devices on smithi161 2023-07-14T19:26:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi161 to 4292M 2023-07-14T19:26:16.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2023-07-14T19:26:16.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: osdmap e75: 12 total, 11 up, 12 in 2023-07-14T19:26:16.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:16.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:16 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:17.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/1070652451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40fec40 msgr2=0x7f3bc40ff050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/1070652451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40fec40 0x7f3bc40ff050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009990 tx=0x7f3bc0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/1070652451 shutdown_connections 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/1070652451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc41007d0 0x7f3bc4107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/1070652451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40fec40 0x7f3bc40ff050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/1070652451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc40ffe40 0x7f3bc4100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/1070652451 >> 172.21.15.82:0/1070652451 conn(0x7f3bc40fa290 msgr2=0x7f3bc40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:17.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/1070652451 shutdown_connections 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.358+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/1070652451 wait complete. 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 Processor -- start 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 -- start start 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc40fec40 0x7f3bc4197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc40ffe40 0x7f3bc41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 0x7f3bc419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:17.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419f7c0 con 0x7f3bc41007d0 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419f900 con 0x7f3bc40ffe40 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcc7f1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419fac0 con 0x7f3bc40fec40 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 0x7f3bc419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 0x7f3bc419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40364/0 (socket says 172.21.15.82:40364) 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 -- 172.21.15.82:0/851067367 learned_addr learned my addr 172.21.15.82:0/851067367 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bc9d8c700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc40ffe40 0x7f3bc41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bca58d700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc40fec40 0x7f3bc4197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:17.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 -- 172.21.15.82:0/851067367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc40fec40 msgr2=0x7f3bc4197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:17.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc40fec40 0x7f3bc4197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 -- 172.21.15.82:0/851067367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc40ffe40 msgr2=0x7f3bc41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:17.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc40ffe40 0x7f3bc41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.359+0000 7f3bcad8e700 1 -- 172.21.15.82:0/851067367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc00095f0 con 0x7f3bc41007d0 2023-07-14T19:26:17.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.360+0000 7f3bcad8e700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 0x7f3bc419f1f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc007ee0 tx=0x7f3bbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.360+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc017070 con 0x7f3bc41007d0 2023-07-14T19:26:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.360+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc419fda0 con 0x7f3bc41007d0 2023-07-14T19:26:17.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.360+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bbc00f040 con 0x7f3bc41007d0 2023-07-14T19:26:17.366 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.360+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc005740 con 0x7f3bc41007d0 2023-07-14T19:26:17.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.360+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc41a01e0 con 0x7f3bc41007d0 2023-07-14T19:26:17.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.361+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc410a4f0 con 0x7f3bc41007d0 2023-07-14T19:26:17.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.362+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3bbc00dc60 con 0x7f3bc41007d0 2023-07-14T19:26:17.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.362+0000 7f3bbb7fe700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bb00a1150 0x7f3bb00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:17.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.362+0000 7f3bca58d700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bb00a1150 0x7f3bb00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:17.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.362+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3bbc0cbc80 con 0x7f3bc41007d0 2023-07-14T19:26:17.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.363+0000 7f3bca58d700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bb00a1150 0x7f3bb00a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006010 tx=0x7f3bc000be00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:17.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.367+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3bbc095870 con 0x7f3bc41007d0 2023-07-14T19:26:17.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:17 smithi133 ceph-mon[128894]: purged_snaps scrub starts 2023-07-14T19:26:17.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:17 smithi133 ceph-mon[128894]: purged_snaps scrub ok 2023-07-14T19:26:17.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:17 smithi133 ceph-mon[128894]: osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] boot 2023-07-14T19:26:17.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:17 smithi133 ceph-mon[128894]: osdmap e76: 12 total, 12 up, 12 in 2023-07-14T19:26:17.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:17 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:17.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:17 smithi082 ceph-mon[127461]: purged_snaps scrub starts 2023-07-14T19:26:17.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:17 smithi082 ceph-mon[127461]: purged_snaps scrub ok 2023-07-14T19:26:17.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:17 smithi082 ceph-mon[127461]: osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] boot 2023-07-14T19:26:17.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:17 smithi082 ceph-mon[127461]: osdmap e76: 12 total, 12 up, 12 in 2023-07-14T19:26:17.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:17 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:17.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:17 smithi161 ceph-mon[129184]: purged_snaps scrub starts 2023-07-14T19:26:17.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:17 smithi161 ceph-mon[129184]: purged_snaps scrub ok 2023-07-14T19:26:17.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:17 smithi161 ceph-mon[129184]: osd.11 [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] boot 2023-07-14T19:26:17.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:17 smithi161 ceph-mon[129184]: osdmap e76: 12 total, 12 up, 12 in 2023-07-14T19:26:17.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:17 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:26:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.552+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f3bc41a07f0 con 0x7f3bc41007d0 2023-07-14T19:26:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.553+0000 7f3bbb7fe700 1 -- 172.21.15.82:0/851067367 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f3bbc098e90 con 0x7f3bc41007d0 2023-07-14T19:26:17.554 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bb00a1150 msgr2=0x7f3bb00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:17.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bb00a1150 0x7f3bb00a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0006010 tx=0x7f3bc000be00 comp rx=0 tx=0).stop 2023-07-14T19:26:17.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 msgr2=0x7f3bc419f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:17.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 0x7f3bc419f1f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc007ee0 tx=0x7f3bbc00d3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 shutdown_connections 2023-07-14T19:26:17.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bb00a1150 0x7f3bb00a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc41007d0 0x7f3bc419f1f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bc40fec40 0x7f3bc4197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 --2- 172.21.15.82:0/851067367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc40ffe40 0x7f3bc41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:17.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 >> 172.21.15.82:0/851067367 conn(0x7f3bc40fa290 msgr2=0x7f3bc4068ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:17.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 shutdown_connections 2023-07-14T19:26:17.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:17.555+0000 7f3bcc7f1700 1 -- 172.21.15.82:0/851067367 wait complete. 2023-07-14T19:26:18.127 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:18 smithi082 ceph-mon[127461]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:18.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:18 smithi082 ceph-mon[127461]: osdmap e77: 12 total, 12 up, 12 in 2023-07-14T19:26:18.128 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:18 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/851067367' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T19:26:18.129 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1689362776,"num_in_osds":12,"osd_in_since":1689362758,"num_remapped_pgs":0} 2023-07-14T19:26:18.129 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T19:26:18.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:18 smithi133 ceph-mon[128894]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:18.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:18 smithi133 ceph-mon[128894]: osdmap e77: 12 total, 12 up, 12 in 2023-07-14T19:26:18.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:18 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/851067367' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T19:26:18.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:18 smithi161 ceph-mon[129184]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-14T19:26:18.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:18 smithi161 ceph-mon[129184]: osdmap e77: 12 total, 12 up, 12 in 2023-07-14T19:26:18.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:18 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/851067367' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-14T19:26:18.682 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:26:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 -- 172.21.15.82:0/4115144171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 msgr2=0x7f6068103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 --2- 172.21.15.82:0/4115144171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f6068103f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60580099c0 tx=0x7f6058009cd0 comp rx=0 tx=0).stop 2023-07-14T19:26:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 -- 172.21.15.82:0/4115144171 shutdown_connections 2023-07-14T19:26:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 --2- 172.21.15.82:0/4115144171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60680ffd50 0x7f606810b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 --2- 172.21.15.82:0/4115144171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f6068103f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 --2- 172.21.15.82:0/4115144171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 0x7f60680ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 -- 172.21.15.82:0/4115144171 >> 172.21.15.82:0/4115144171 conn(0x7f6068075280 msgr2=0x7f6068075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 -- 172.21.15.82:0/4115144171 shutdown_connections 2023-07-14T19:26:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.845+0000 7f606fee8700 1 -- 172.21.15.82:0/4115144171 wait complete. 2023-07-14T19:26:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 Processor -- start 2023-07-14T19:26:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 -- start start 2023-07-14T19:26:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 0x7f6068196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60680ffd50 0x7f6068197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f60681a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60681a21b0 con 0x7f6068103b60 2023-07-14T19:26:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60681a22f0 con 0x7f60680ffd50 2023-07-14T19:26:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606fee8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60681a2430 con 0x7f60680ff3a0 2023-07-14T19:26:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606e485700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f60681a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606e485700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f60681a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40378/0 (socket says 172.21.15.82:40378) 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.846+0000 7f606e485700 1 -- 172.21.15.82:0/1875586615 learned_addr learned my addr 172.21.15.82:0/1875586615 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606d483700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60680ffd50 0x7f6068197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606dc84700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 0x7f6068196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606e485700 1 -- 172.21.15.82:0/1875586615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 msgr2=0x7f6068196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606e485700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 0x7f6068196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606e485700 1 -- 172.21.15.82:0/1875586615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60680ffd50 msgr2=0x7f6068197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606e485700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60680ffd50 0x7f6068197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606e485700 1 -- 172.21.15.82:0/1875586615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580095f0 con 0x7f6068103b60 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606dc84700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 0x7f6068196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606e485700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f60681a1c70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f606000ef30 tx=0x7f606000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060018070 con 0x7f6068103b60 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606000f040 con 0x7f6068103b60 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60681a2630 con 0x7f6068103b60 2023-07-14T19:26:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606000cdb0 con 0x7f6068103b60 2023-07-14T19:26:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.847+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60681a2b30 con 0x7f6068103b60 2023-07-14T19:26:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.848+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606804e340 con 0x7f6068103b60 2023-07-14T19:26:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.849+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6060003a00 con 0x7f6068103b60 2023-07-14T19:26:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.849+0000 7f605effd700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60540a11d0 0x7f60540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.850+0000 7f606dc84700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60540a11d0 0x7f60540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.850+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f60600cc940 con 0x7f6068103b60 2023-07-14T19:26:19.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.850+0000 7f606dc84700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60540a11d0 0x7f60540a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6058009990 tx=0x7f605801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:19.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:19.853+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f60600965c0 con 0x7f6068103b60 2023-07-14T19:26:20.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.039+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60681a2e50 con 0x7f6068103b60 2023-07-14T19:26:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.040+0000 7f605effd700 1 -- 172.21.15.82:0/1875586615 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18432 (secure 0 0 0) 0x7f6060099be0 con 0x7f6068103b60 2023-07-14T19:26:20.041 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:20.041 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:26:17.073692+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60540a11d0 msgr2=0x7f60540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60540a11d0 0x7f60540a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6058009990 tx=0x7f605801a040 comp rx=0 tx=0).stop 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 msgr2=0x7f60681a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f60681a1c70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f606000ef30 tx=0x7f606000c3a0 comp rx=0 tx=0).stop 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 shutdown_connections 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60540a11d0 0x7f60540a3680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6068103b60 0x7f60681a1c70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60680ff3a0 0x7f6068196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 --2- 172.21.15.82:0/1875586615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60680ffd50 0x7f6068197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 >> 172.21.15.82:0/1875586615 conn(0x7f6068075280 msgr2=0x7f60680fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 shutdown_connections 2023-07-14T19:26:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:20.042+0000 7f606fee8700 1 -- 172.21.15.82:0/1875586615 wait complete. 2023-07-14T19:26:20.425 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:20 smithi082 ceph-mon[127461]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:20.426 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:20 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1875586615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:20.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:20 smithi133 ceph-mon[128894]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:20.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:20 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1875586615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:20.447 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:26:20.448 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-14T19:26:20.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:20 smithi161 ceph-mon[129184]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:20.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:20 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1875586615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:21.002 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:26:22.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 -- 172.21.15.82:0/613200827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 msgr2=0x7f786410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:22.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 --2- 172.21.15.82:0/613200827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f786410ba60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b0d0 tx=0x7f785c00b3e0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 -- 172.21.15.82:0/613200827 shutdown_connections 2023-07-14T19:26:22.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 --2- 172.21.15.82:0/613200827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f786410ba60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.177 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 --2- 172.21.15.82:0/613200827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78640fee40 0x7f78640ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 --2- 172.21.15.82:0/613200827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78640ff7e0 0x7f78641037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 -- 172.21.15.82:0/613200827 >> 172.21.15.82:0/613200827 conn(0x7f7864075ca0 msgr2=0x7f78640760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:22.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 -- 172.21.15.82:0/613200827 shutdown_connections 2023-07-14T19:26:22.178 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 -- 172.21.15.82:0/613200827 wait complete. 2023-07-14T19:26:22.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.173+0000 7f78696d3700 1 Processor -- start 2023-07-14T19:26:22.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 -- start start 2023-07-14T19:26:22.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78640fee40 0x7f7864196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:22.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78640ff7e0 0x7f7864197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:22.179 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f78641a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:22.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864197a80 con 0x7f7864103ee0 2023-07-14T19:26:22.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78641a21f0 con 0x7f78640fee40 2023-07-14T19:26:22.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78641a2360 con 0x7f78640ff7e0 2023-07-14T19:26:22.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f78641a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:22.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f78641a1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40390/0 (socket says 172.21.15.82:40390) 2023-07-14T19:26:22.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 -- 172.21.15.82:0/3602552477 learned_addr learned my addr 172.21.15.82:0/3602552477 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:22.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f7862ffd700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78640fee40 0x7f7864196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:22.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78627fc700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78640ff7e0 0x7f7864197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:22.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 -- 172.21.15.82:0/3602552477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78640ff7e0 msgr2=0x7f7864197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:22.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78640ff7e0 0x7f7864197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 -- 172.21.15.82:0/3602552477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78640fee40 msgr2=0x7f7864196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:22.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78640fee40 0x7f7864196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 -- 172.21.15.82:0/3602552477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c009d00 con 0x7f7864103ee0 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78637fe700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f78641a1cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f785c007870 tx=0x7f785c0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00e040 con 0x7f7864103ee0 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.174+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78641a25e0 con 0x7f7864103ee0 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.175+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f785c003e10 con 0x7f7864103ee0 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.175+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c01b8a0 con 0x7f7864103ee0 2023-07-14T19:26:22.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.175+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78641a2a50 con 0x7f7864103ee0 2023-07-14T19:26:22.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.176+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786404e340 con 0x7f7864103ee0 2023-07-14T19:26:22.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.177+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f785c01ba60 con 0x7f7864103ee0 2023-07-14T19:26:22.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.178+0000 7f784bfff700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f784c0a1150 0x7f784c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:22.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.178+0000 7f7862ffd700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f784c0a1150 0x7f784c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:22.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.178+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f785c0ce2f0 con 0x7f7864103ee0 2023-07-14T19:26:22.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.178+0000 7f7862ffd700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f784c0a1150 0x7f784c0a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f785800ac50 tx=0x7f785800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:22.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.181+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f785c097ed0 con 0x7f7864103ee0 2023-07-14T19:26:22.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.368+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78641a2c70 con 0x7f7864103ee0 2023-07-14T19:26:22.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.368+0000 7f784bfff700 1 -- 172.21.15.82:0/3602552477 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f785c09b4f0 con 0x7f7864103ee0 2023-07-14T19:26:22.369 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:26:22.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f784c0a1150 msgr2=0x7f784c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:22.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f784c0a1150 0x7f784c0a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f785800ac50 tx=0x7f785800a380 comp rx=0 tx=0).stop 2023-07-14T19:26:22.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 msgr2=0x7f78641a1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:22.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f78641a1cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f785c007870 tx=0x7f785c0078a0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 shutdown_connections 2023-07-14T19:26:22.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f784c0a1150 0x7f784c0a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864103ee0 0x7f78641a1cb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78640fee40 0x7f7864196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 --2- 172.21.15.82:0/3602552477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78640ff7e0 0x7f7864197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:22.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.370+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 >> 172.21.15.82:0/3602552477 conn(0x7f7864075ca0 msgr2=0x7f78640fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:22.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.371+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 shutdown_connections 2023-07-14T19:26:22.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:22.371+0000 7f78696d3700 1 -- 172.21.15.82:0/3602552477 wait complete. 2023-07-14T19:26:22.392 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:22 smithi082 ceph-mon[127461]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:22.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:22 smithi133 ceph-mon[128894]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:22.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:22 smithi161 ceph-mon[129184]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:22.758 INFO:tasks.cephadm:Adding mds.a on smithi082 2023-07-14T19:26:22.758 INFO:tasks.cephadm:Adding mds.d on smithi082 2023-07-14T19:26:22.759 INFO:tasks.cephadm:Adding mds.g on smithi082 2023-07-14T19:26:22.759 INFO:tasks.cephadm:Adding mds.j on smithi082 2023-07-14T19:26:22.759 INFO:tasks.cephadm:Adding mds.b on smithi133 2023-07-14T19:26:22.759 INFO:tasks.cephadm:Adding mds.e on smithi133 2023-07-14T19:26:22.759 INFO:tasks.cephadm:Adding mds.h on smithi133 2023-07-14T19:26:22.760 INFO:tasks.cephadm:Adding mds.k on smithi133 2023-07-14T19:26:22.760 INFO:tasks.cephadm:Adding mds.c on smithi161 2023-07-14T19:26:22.760 INFO:tasks.cephadm:Adding mds.f on smithi161 2023-07-14T19:26:22.760 INFO:tasks.cephadm:Adding mds.i on smithi161 2023-07-14T19:26:22.761 INFO:tasks.cephadm:Adding mds.l on smithi161 2023-07-14T19:26:22.761 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph orch apply mds all '12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2023-07-14T19:26:23.329 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:26:23.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:23 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3602552477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:23.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:23 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3602552477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:23.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:23 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3602552477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:24.178 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:24 smithi161 ceph-mon[129184]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:24.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:24 smithi133 ceph-mon[128894]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:24.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:24 smithi082 ceph-mon[127461]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2055761515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc0febb0 msgr2=0x7f4ecc0fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2055761515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc0fefc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009990 tx=0x7f4ebc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2055761515 shutdown_connections 2023-07-14T19:26:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2055761515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc100740 0x7f4ecc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.612 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2055761515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc0fefc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2055761515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0ffdb0 0x7f4ecc100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2055761515 >> 172.21.15.161:0/2055761515 conn(0x7f4ecc0fa1e0 msgr2=0x7f4ecc0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2055761515 shutdown_connections 2023-07-14T19:26:24.613 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.611+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2055761515 wait complete. 2023-07-14T19:26:24.614 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 Processor -- start 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 -- start start 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0ffdb0 0x7f4ecc195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 0x7f4ecc19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19d510 con 0x7f4ecc0febb0 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19d650 con 0x7f4ecc0ffdb0 2023-07-14T19:26:24.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed25d0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc19d7e0 con 0x7f4ecc100740 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed0b6d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 0x7f4ecc19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed0b6d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 0x7f4ecc19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:60594/0 (socket says 172.21.15.161:60594) 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ed0b6d700 1 -- 172.21.15.161:0/2356383843 learned_addr learned my addr 172.21.15.161:0/2356383843 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ecbfff700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.612+0000 7f4ecb7fe700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0ffdb0 0x7f4ecc195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed0b6d700 1 -- 172.21.15.161:0/2356383843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0ffdb0 msgr2=0x7f4ecc195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed0b6d700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0ffdb0 0x7f4ecc195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.616 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed0b6d700 1 -- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0febb0 msgr2=0x7f4ecc195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed0b6d700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed0b6d700 1 -- 172.21.15.161:0/2356383843 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0095f0 con 0x7f4ecc100740 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ecbfff700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed0b6d700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 0x7f4ecc19cf40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400add0 tx=0x7f4ec400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400f070 con 0x7f4ecc100740 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec4007b30 con 0x7f4ecc100740 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc19dac0 con 0x7f4ecc100740 2023-07-14T19:26:24.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40053e0 con 0x7f4ecc100740 2023-07-14T19:26:24.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.613+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc19df70 con 0x7f4ecc100740 2023-07-14T19:26:24.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.614+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc106140 con 0x7f4ecc100740 2023-07-14T19:26:24.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.615+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4ec400c940 con 0x7f4ecc100740 2023-07-14T19:26:24.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.616+0000 7f4ec97fa700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4eb40a1150 0x7f4eb40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:24.619 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.616+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4ec40cbc30 con 0x7f4ecc100740 2023-07-14T19:26:24.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.616+0000 7f4ecbfff700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4eb40a1150 0x7f4eb40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:24.620 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.616+0000 7f4ecbfff700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4eb40a1150 0x7f4eb40a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006010 tx=0x7f4ebc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:24.621 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.620+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4ec4095810 con 0x7f4ecc100740 2023-07-14T19:26:24.816 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.816+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ecc060d40 con 0x7f4eb40a1150 2023-07-14T19:26:24.824 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.824+0000 7f4ec97fa700 1 -- 172.21.15.161:0/2356383843 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f4ecc060d40 con 0x7f4eb40a1150 2023-07-14T19:26:24.825 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2023-07-14T19:26:24.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4eb40a1150 msgr2=0x7f4eb40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:24.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4eb40a1150 0x7f4eb40a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc006010 tx=0x7f4ebc00b540 comp rx=0 tx=0).stop 2023-07-14T19:26:24.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 msgr2=0x7f4ecc19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:24.827 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 0x7f4ecc19cf40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400add0 tx=0x7f4ec400c330 comp rx=0 tx=0).stop 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 shutdown_connections 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ecc100740 0x7f4ecc19cf40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4eb40a1150 0x7f4eb40a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ecc0febb0 0x7f4ecc195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 --2- 172.21.15.161:0/2356383843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0ffdb0 0x7f4ecc195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.826+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 >> 172.21.15.161:0/2356383843 conn(0x7f4ecc0fa1e0 msgr2=0x7f4ecc103280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:24.828 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.827+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 shutdown_connections 2023-07-14T19:26:24.829 INFO:teuthology.orchestra.run.smithi161.stderr:2023-07-14T19:26:24.827+0000 7f4ed25d0700 1 -- 172.21.15.161:0/2356383843 wait complete. 2023-07-14T19:26:25.282 DEBUG:teuthology.orchestra.run.smithi082:mds.a> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.a.service 2023-07-14T19:26:25.285 DEBUG:teuthology.orchestra.run.smithi082:mds.d> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.d.service 2023-07-14T19:26:25.321 INFO:journalctl@ceph.mds.a.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:26:25.322 DEBUG:teuthology.orchestra.run.smithi082:mds.g> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.g.service 2023-07-14T19:26:25.352 INFO:journalctl@ceph.mds.d.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:26:25.353 DEBUG:teuthology.orchestra.run.smithi082:mds.j> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.j.service 2023-07-14T19:26:25.380 INFO:journalctl@ceph.mds.g.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:26:25.381 DEBUG:teuthology.orchestra.run.smithi133:mds.b> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.b.service 2023-07-14T19:26:25.384 DEBUG:teuthology.orchestra.run.smithi133:mds.e> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.e.service 2023-07-14T19:26:25.408 INFO:journalctl@ceph.mds.j.smithi082.stdout:-- Logs begin at Fri 2023-07-14 18:56:07 UTC. -- 2023-07-14T19:26:25.423 INFO:journalctl@ceph.mds.b.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:26:25.424 DEBUG:teuthology.orchestra.run.smithi133:mds.h> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.h.service 2023-07-14T19:26:25.452 INFO:journalctl@ceph.mds.e.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:26:25.453 DEBUG:teuthology.orchestra.run.smithi133:mds.k> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.k.service 2023-07-14T19:26:25.480 INFO:journalctl@ceph.mds.h.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:26:25.481 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service 2023-07-14T19:26:25.483 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.f.service 2023-07-14T19:26:25.507 INFO:journalctl@ceph.mds.k.smithi133.stdout:-- Logs begin at Fri 2023-07-14 18:55:04 UTC. -- 2023-07-14T19:26:25.513 INFO:journalctl@ceph.mds.c.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:26:25.514 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.i.service 2023-07-14T19:26:25.541 INFO:journalctl@ceph.mds.f.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:26:25.542 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.l.service 2023-07-14T19:26:25.568 INFO:journalctl@ceph.mds.i.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:26:25.570 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-14T19:26:25.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:26:25.599 INFO:journalctl@ceph.mds.l.smithi161.stdout:-- Logs begin at Fri 2023-07-14 18:55:15 UTC. -- 2023-07-14T19:26:25.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 -- 172.21.15.82:0/2249489383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a80ffc30 msgr2=0x7f13a8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:25.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 --2- 172.21.15.82:0/2249489383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a80ffc30 0x7f13a8100040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13a000b0a0 tx=0x7f13a000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 -- 172.21.15.82:0/2249489383 shutdown_connections 2023-07-14T19:26:25.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 --2- 172.21.15.82:0/2249489383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a81017b0 0x7f13a8108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 --2- 172.21.15.82:0/2249489383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a80ffc30 0x7f13a8100040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 --2- 172.21.15.82:0/2249489383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13a8100e20 0x7f13a8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 -- 172.21.15.82:0/2249489383 >> 172.21.15.82:0/2249489383 conn(0x7f13a80fb230 msgr2=0x7f13a80fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:25.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 -- 172.21.15.82:0/2249489383 shutdown_connections 2023-07-14T19:26:25.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.748+0000 7f13afcfd700 1 -- 172.21.15.82:0/2249489383 wait complete. 2023-07-14T19:26:25.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 Processor -- start 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 -- start start 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a80ffc30 0x7f13a8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13a8100e20 0x7f13a81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 0x7f13a819f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a819f800 con 0x7f13a81017b0 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a819f970 con 0x7f13a80ffc30 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13afcfd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a819fb30 con 0x7f13a8100e20 2023-07-14T19:26:25.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 0x7f13a819f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 0x7f13a819f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40404/0 (socket says 172.21.15.82:40404) 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 -- 172.21.15.82:0/3372866487 learned_addr learned my addr 172.21.15.82:0/3372866487 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ad298700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13a8100e20 0x7f13a81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ada99700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a80ffc30 0x7f13a8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 -- 172.21.15.82:0/3372866487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13a8100e20 msgr2=0x7f13a81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13a8100e20 0x7f13a81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 -- 172.21.15.82:0/3372866487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a80ffc30 msgr2=0x7f13a8197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a80ffc30 0x7f13a8197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ae29a700 1 -- 172.21.15.82:0/3372866487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0009d00 con 0x7f13a81017b0 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.749+0000 7f13ada99700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a80ffc30 0x7f13a8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.750+0000 7f13ae29a700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 0x7f13a819f230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13a400d980 tx=0x7f13a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.750+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004a30 con 0x7f13a81017b0 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.750+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a400b5c0 con 0x7f13a81017b0 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.750+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4009b50 con 0x7f13a81017b0 2023-07-14T19:26:25.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.750+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a819fe10 con 0x7f13a81017b0 2023-07-14T19:26:25.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.750+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a81a0290 con 0x7f13a81017b0 2023-07-14T19:26:25.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.751+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a810b400 con 0x7f13a81017b0 2023-07-14T19:26:25.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.752+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f13a4009cb0 con 0x7f13a81017b0 2023-07-14T19:26:25.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.752+0000 7f139effd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f13940a1220 0x7f13940a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:25.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.752+0000 7f13ada99700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f13940a1220 0x7f13940a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:25.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.752+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13a40cc630 con 0x7f13a81017b0 2023-07-14T19:26:25.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.753+0000 7f13ada99700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f13940a1220 0x7f13940a36d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009fd0 tx=0x7f13a0006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:25.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.756+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f13a40962a0 con 0x7f13a81017b0 2023-07-14T19:26:25.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.946+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13a81a08d0 con 0x7f13a81017b0 2023-07-14T19:26:25.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.947+0000 7f139effd700 1 -- 172.21.15.82:0/3372866487 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18432 (secure 0 0 0) 0x7f13a4017020 con 0x7f13a81017b0 2023-07-14T19:26:25.950 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:25.950 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:26:17.073692+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:26:25.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f13940a1220 msgr2=0x7f13940a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:25.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f13940a1220 0x7f13940a36d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009fd0 tx=0x7f13a0006230 comp rx=0 tx=0).stop 2023-07-14T19:26:25.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 msgr2=0x7f13a819f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 0x7f13a819f230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13a400d980 tx=0x7f13a400dc90 comp rx=0 tx=0).stop 2023-07-14T19:26:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 shutdown_connections 2023-07-14T19:26:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f13940a1220 0x7f13940a36d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13a81017b0 0x7f13a819f230 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13a80ffc30 0x7f13a8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 --2- 172.21.15.82:0/3372866487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f13a8100e20 0x7f13a81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:25.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 >> 172.21.15.82:0/3372866487 conn(0x7f13a80fb230 msgr2=0x7f13a80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:25.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 shutdown_connections 2023-07-14T19:26:25.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:25.949+0000 7f13afcfd700 1 -- 172.21.15.82:0/3372866487 wait complete. 2023-07-14T19:26:25.962 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:25.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:26:25.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2023-07-14T19:26:25.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:25.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:25.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:25.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:25.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:25 smithi082 ceph-mon[127461]: Deploying daemon mds.l on smithi161 2023-07-14T19:26:25.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:26:25.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:26:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 -- 172.21.15.82:0/3978123164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2341006f0 msgr2=0x7fe234102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 --2- 172.21.15.82:0/3978123164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2341006f0 0x7fe234102ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe228009990 tx=0x7fe228009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 -- 172.21.15.82:0/3978123164 shutdown_connections 2023-07-14T19:26:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 --2- 172.21.15.82:0/3978123164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe234105930 0x7fe234107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 --2- 172.21.15.82:0/3978123164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2341006f0 0x7fe234102ad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 --2- 172.21.15.82:0/3978123164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe234103010 0x7fe2341053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 -- 172.21.15.82:0/3978123164 >> 172.21.15.82:0/3978123164 conn(0x7fe2340fa130 msgr2=0x7fe2340fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 -- 172.21.15.82:0/3978123164 shutdown_connections 2023-07-14T19:26:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 -- 172.21.15.82:0/3978123164 wait complete. 2023-07-14T19:26:26.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.089+0000 7fe2396b5700 1 Processor -- start 2023-07-14T19:26:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 -- start start 2023-07-14T19:26:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2341006f0 0x7fe234195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 0x7fe234195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe234105930 0x7fe23419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419d500 con 0x7fe234103010 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419d640 con 0x7fe2341006f0 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23419d7d0 con 0x7fe234105930 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 0x7fe234195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 0x7fe234195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40416/0 (socket says 172.21.15.82:40416) 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 -- 172.21.15.82:0/1571239010 learned_addr learned my addr 172.21.15.82:0/1571239010 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2337fe700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe234105930 0x7fe23419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe232ffd700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2341006f0 0x7fe234195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 -- 172.21.15.82:0/1571239010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe234105930 msgr2=0x7fe23419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe234105930 0x7fe23419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 -- 172.21.15.82:0/1571239010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2341006f0 msgr2=0x7fe234195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2341006f0 0x7fe234195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 -- 172.21.15.82:0/1571239010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280095f0 con 0x7fe234103010 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe232ffd700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2341006f0 0x7fe234195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:26:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2327fc700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 0x7fe234195ec0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9a0 tx=0x7fe22400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:26.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224009790 con 0x7fe234103010 2023-07-14T19:26:26.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe22400cbd0 con 0x7fe234103010 2023-07-14T19:26:26.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23419dab0 con 0x7fe234103010 2023-07-14T19:26:26.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224018680 con 0x7fe234103010 2023-07-14T19:26:26.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.090+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23419dfc0 con 0x7fe234103010 2023-07-14T19:26:26.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.091+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2340fbca0 con 0x7fe234103010 2023-07-14T19:26:26.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.092+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe22400cd40 con 0x7fe234103010 2023-07-14T19:26:26.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.093+0000 7fe21bfff700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe21c0a1150 0x7fe21c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.093+0000 7fe232ffd700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe21c0a1150 0x7fe21c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.093+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe224014070 con 0x7fe234103010 2023-07-14T19:26:26.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.093+0000 7fe232ffd700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe21c0a1150 0x7fe21c0a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe22800b2e0 tx=0x7fe228000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:26.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.097+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe224051780 con 0x7fe234103010 2023-07-14T19:26:26.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:26:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2023-07-14T19:26:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:26.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:26.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:26.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:26.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:26.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:26.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:25 smithi133 ceph-mon[128894]: Deploying daemon mds.l on smithi161 2023-07-14T19:26:26.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:26.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:26:26.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: Saving service mds.all spec with placement smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2023-07-14T19:26:26.203 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:26.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:26.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:26.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:26.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:26.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:26.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:26.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:26.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:25 smithi161 ceph-mon[129184]: Deploying daemon mds.l on smithi161 2023-07-14T19:26:26.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.283+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe23402caf0 con 0x7fe234103010 2023-07-14T19:26:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.284+0000 7fe21bfff700 1 -- 172.21.15.82:0/1571239010 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2240962b0 con 0x7fe234103010 2023-07-14T19:26:26.286 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:26:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe21c0a1150 msgr2=0x7fe21c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe21c0a1150 0x7fe21c0a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe22800b2e0 tx=0x7fe228000bc0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 msgr2=0x7fe234195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 0x7fe234195ec0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe22400e9a0 tx=0x7fe22400ecb0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 shutdown_connections 2023-07-14T19:26:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe234105930 0x7fe23419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2341006f0 0x7fe234195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe21c0a1150 0x7fe21c0a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 --2- 172.21.15.82:0/1571239010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe234103010 0x7fe234195ec0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 >> 172.21.15.82:0/1571239010 conn(0x7fe2340fa130 msgr2=0x7fe234101350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 shutdown_connections 2023-07-14T19:26:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.286+0000 7fe2396b5700 1 -- 172.21.15.82:0/1571239010 wait complete. 2023-07-14T19:26:26.301 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 -- 172.21.15.82:0/1051421608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 msgr2=0x7f4120100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 --2- 172.21.15.82:0/1051421608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120100140 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f410c009990 tx=0x7f410c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 -- 172.21.15.82:0/1051421608 shutdown_connections 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 --2- 172.21.15.82:0/1051421608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4120100680 0x7f4120106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 --2- 172.21.15.82:0/1051421608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41200feb00 0x7f41200fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 --2- 172.21.15.82:0/1051421608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120100140 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 -- 172.21.15.82:0/1051421608 >> 172.21.15.82:0/1051421608 conn(0x7f41200fa100 msgr2=0x7f41200fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 -- 172.21.15.82:0/1051421608 shutdown_connections 2023-07-14T19:26:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 -- 172.21.15.82:0/1051421608 wait complete. 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 Processor -- start 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.422+0000 7f4126d39700 1 -- start start 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41200feb00 0x7f41201947f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4120100680 0x7f412019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412019c370 con 0x7f41200ffcf0 2023-07-14T19:26:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412019c4b0 con 0x7f4120100680 2023-07-14T19:26:26.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412019c640 con 0x7f41200feb00 2023-07-14T19:26:26.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40430/0 (socket says 172.21.15.82:40430) 2023-07-14T19:26:26.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 -- 172.21.15.82:0/543350508 learned_addr learned my addr 172.21.15.82:0/543350508 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:26.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4124ad5700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41200feb00 0x7f41201947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f41252d6700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4120100680 0x7f412019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 -- 172.21.15.82:0/543350508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41200feb00 msgr2=0x7f41201947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41200feb00 0x7f41201947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 -- 172.21.15.82:0/543350508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4120100680 msgr2=0x7f412019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4120100680 0x7f412019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 -- 172.21.15.82:0/543350508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0095f0 con 0x7f41200ffcf0 2023-07-14T19:26:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411ffff700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120194d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f410c00b620 tx=0x7f410c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c01f070 con 0x7f41200ffcf0 2023-07-14T19:26:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f410c005030 con 0x7f41200ffcf0 2023-07-14T19:26:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41200677a0 con 0x7f41200ffcf0 2023-07-14T19:26:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.423+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c004bf0 con 0x7f41200ffcf0 2023-07-14T19:26:26.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.424+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4120067bf0 con 0x7f41200ffcf0 2023-07-14T19:26:26.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.425+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120068640 con 0x7f41200ffcf0 2023-07-14T19:26:26.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.426+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f410c023070 con 0x7f41200ffcf0 2023-07-14T19:26:26.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.426+0000 7f411dffb700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41100a11a0 0x7f41100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:26.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.426+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f410c0cfda0 con 0x7f41200ffcf0 2023-07-14T19:26:26.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.426+0000 7f4124ad5700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41100a11a0 0x7f41100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:26.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.427+0000 7f4124ad5700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41100a11a0 0x7f41100a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41200ffb50 tx=0x7f4114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:26.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.430+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f410c099a20 con 0x7f41200ffcf0 2023-07-14T19:26:26.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.662+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4120105a10 con 0x7f41200ffcf0 2023-07-14T19:26:26.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.662+0000 7f411dffb700 1 -- 172.21.15.82:0/543350508 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f410c09d040 con 0x7f41200ffcf0 2023-07-14T19:26:26.664 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:26.664 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-14T19:26:26.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41100a11a0 msgr2=0x7f41100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41100a11a0 0x7f41100a3650 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41200ffb50 tx=0x7f4114009260 comp rx=0 tx=0).stop 2023-07-14T19:26:26.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 msgr2=0x7f4120194d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:26.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120194d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f410c00b620 tx=0x7f410c004730 comp rx=0 tx=0).stop 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 shutdown_connections 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4120100680 0x7f412019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41200feb00 0x7f41201947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f41100a11a0 0x7f41100a3650 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 --2- 172.21.15.82:0/543350508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41200ffcf0 0x7f4120194d30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 >> 172.21.15.82:0/543350508 conn(0x7f41200fa100 msgr2=0x7f4120103170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 shutdown_connections 2023-07-14T19:26:26.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:26.664+0000 7f4126d39700 1 -- 172.21.15.82:0/543350508 wait complete. 2023-07-14T19:26:26.668 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 1 2023-07-14T19:26:26.741 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:26:26 smithi161 systemd[1]: Starting Ceph mds.l for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:26.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3372866487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:26.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1571239010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:26.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/543350508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:27.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:26 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3372866487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:27.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:26 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1571239010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:27.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:26 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/543350508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:27.131 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:26:26 smithi161 podman[154267]: 2023-07-14T19:26:27.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:26 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3372866487' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:27.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:26 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1571239010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:27.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:26 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/543350508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:27.491 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:26:27 smithi161 bash[154267]: 74519dae9ef334ed810f27d819d4117accad3a73e0f44bc0f6f75c0188d2c45a 2023-07-14T19:26:27.491 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mds[154307]: -- 172.21.15.161:0/1261447715 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556037a011e0 con 0x5560379d7000 2023-07-14T19:26:27.492 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-l[154283]: starting mds.l at 2023-07-14T19:26:27.492 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:26:27 smithi161 systemd[1]: Started Ceph mds.l for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:28.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:28.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6832/2795796645,v1:172.21.15.161:6833/2795796645] up:boot 2023-07-14T19:26:28.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: fsmap 1 up:standby 2023-07-14T19:26:28.182 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T19:26:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:27 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:28.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:28.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6832/2795796645,v1:172.21.15.161:6833/2795796645] up:boot 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: fsmap 1 up:standby 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:28.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:27 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6832/2795796645,v1:172.21.15.161:6833/2795796645] up:boot 2023-07-14T19:26:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: fsmap 1 up:standby 2023-07-14T19:26:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T19:26:28.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:28.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:28.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:28.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:27 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:28.678 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:28.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- 172.21.15.82:0/3086570010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 msgr2=0x7f7f08101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- 172.21.15.82:0/3086570010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08101260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009990 tx=0x7f7ef4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- 172.21.15.82:0/3086570010 shutdown_connections 2023-07-14T19:26:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- 172.21.15.82:0/3086570010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f081017a0 0x7f7f08108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- 172.21.15.82:0/3086570010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f080ffc20 0x7f7f08100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- 172.21.15.82:0/3086570010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08101260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:28.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- 172.21.15.82:0/3086570010 >> 172.21.15.82:0/3086570010 conn(0x7f7f080fb260 msgr2=0x7f7f080fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- 172.21.15.82:0/3086570010 shutdown_connections 2023-07-14T19:26:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- 172.21.15.82:0/3086570010 wait complete. 2023-07-14T19:26:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 Processor -- start 2023-07-14T19:26:28.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- start start 2023-07-14T19:26:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f080ffc20 0x7f7f08195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f081017a0 0x7f7f0819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819d4d0 con 0x7f7f08100e10 2023-07-14T19:26:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819d610 con 0x7f7f081017a0 2023-07-14T19:26:28.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.812+0000 7f7f0f077700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819d7a0 con 0x7f7f080ffc20 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40444/0 (socket says 172.21.15.82:40444) 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 -- 172.21.15.82:0/1276334741 learned_addr learned my addr 172.21.15.82:0/1276334741 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f0ce13700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f080ffc20 0x7f7f08195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f0d614700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f081017a0 0x7f7f0819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 -- 172.21.15.82:0/1276334741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f080ffc20 msgr2=0x7f7f08195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f080ffc20 0x7f7f08195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:28.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 -- 172.21.15.82:0/1276334741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f081017a0 msgr2=0x7f7f0819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f081017a0 0x7f7f0819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 -- 172.21.15.82:0/1276334741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40095f0 con 0x7f7f08100e10 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f07fff700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08195e90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4004760 tx=0x7f7ef4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef401d070 con 0x7f7f08100e10 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ef4004cf0 con 0x7f7f08100e10 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003d00 con 0x7f7f08100e10 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0819da20 con 0x7f7f08100e10 2023-07-14T19:26:28.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.813+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0819de70 con 0x7f7f08100e10 2023-07-14T19:26:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.814+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0810b490 con 0x7f7f08100e10 2023-07-14T19:26:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.815+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ef4005000 con 0x7f7f08100e10 2023-07-14T19:26:28.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.816+0000 7f7f05ffb700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7ef80a1220 0x7f7ef80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:28.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.816+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7ef40cd980 con 0x7f7f08100e10 2023-07-14T19:26:28.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.816+0000 7f7f0ce13700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7ef80a1220 0x7f7ef80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:28.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.816+0000 7f7f0ce13700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7ef80a1220 0x7f7ef80a36d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f08100c70 tx=0x7f7efc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:28.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:28.819+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7ef4097560 con 0x7f7f08100e10 2023-07-14T19:26:28.835 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:26:28 smithi133 systemd[1]: Starting Ceph mds.k for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:28.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:28 smithi082 ceph-mon[127461]: Deploying daemon mds.k on smithi133 2023-07-14T19:26:29.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.052+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f08106ab0 con 0x7f7f08100e10 2023-07-14T19:26:29.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.052+0000 7f7f05ffb700 1 -- 172.21.15.82:0/1276334741 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f7ef4004e60 con 0x7f7f08100e10 2023-07-14T19:26:29.054 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:29.055 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-14T19:26:29.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.054+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7ef80a1220 msgr2=0x7f7ef80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:29.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.054+0000 7f7f0f077700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7ef80a1220 0x7f7ef80a36d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f08100c70 tx=0x7f7efc009190 comp rx=0 tx=0).stop 2023-07-14T19:26:29.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.054+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 msgr2=0x7f7f08195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.054+0000 7f7f0f077700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08195e90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4004760 tx=0x7f7ef4004790 comp rx=0 tx=0).stop 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 shutdown_connections 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f081017a0 0x7f7f0819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f080ffc20 0x7f7f08195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7ef80a1220 0x7f7ef80a36d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 --2- 172.21.15.82:0/1276334741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f08100e10 0x7f7f08195e90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:29.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 >> 172.21.15.82:0/1276334741 conn(0x7f7f080fb260 msgr2=0x7f7f080fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:29.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 shutdown_connections 2023-07-14T19:26:29.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:29.055+0000 7f7f0f077700 1 -- 172.21.15.82:0/1276334741 wait complete. 2023-07-14T19:26:29.058 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2023-07-14T19:26:29.115 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:28 smithi133 ceph-mon[128894]: Deploying daemon mds.k on smithi133 2023-07-14T19:26:29.115 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:26:29 smithi133 podman[153071]: 2023-07-14T19:26:29.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:28 smithi161 ceph-mon[129184]: Deploying daemon mds.k on smithi133 2023-07-14T19:26:29.683 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:26:29 smithi133 bash[153071]: 6afc4397bcf6be880d7032f6eef4914cccedff3dfb379eb7ba3cb948b127a96a 2023-07-14T19:26:29.683 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mds[153112]: -- 172.21.15.133:0/4272171235 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd964e71e0 con 0x55fd964bc800 2023-07-14T19:26:29.684 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-k[153087]: starting mds.k at 2023-07-14T19:26:29.684 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:26:29 smithi133 systemd[1]: Started Ceph mds.k for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:30.175 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:30.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1276334741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:30.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:30.176 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:30.177 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:29 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:30.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1276334741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:30.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:29 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:30.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:30.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1276334741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:30.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:30.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:30.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:30.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:29 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:31.069 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:31.098 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:26:30 smithi082 systemd[1]: Starting Ceph mds.j for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:31.098 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:30 smithi082 ceph-mon[127461]: Deploying daemon mds.j on smithi082 2023-07-14T19:26:31.098 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:30 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6832/1264023541,v1:172.21.15.133:6833/1264023541] up:boot 2023-07-14T19:26:31.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:30 smithi082 ceph-mon[127461]: fsmap 2 up:standby 2023-07-14T19:26:31.099 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:30 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T19:26:31.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:30 smithi133 ceph-mon[128894]: Deploying daemon mds.j on smithi082 2023-07-14T19:26:31.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:30 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6832/1264023541,v1:172.21.15.133:6833/1264023541] up:boot 2023-07-14T19:26:31.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:30 smithi133 ceph-mon[128894]: fsmap 2 up:standby 2023-07-14T19:26:31.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:30 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T19:26:31.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 -- 172.21.15.82:0/4122818588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c100610 msgr2=0x7fb74c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:31.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 --2- 172.21.15.82:0/4122818588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c100610 0x7fb74c106ed0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb74400b0a0 tx=0x7fb74400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 -- 172.21.15.82:0/4122818588 shutdown_connections 2023-07-14T19:26:31.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 --2- 172.21.15.82:0/4122818588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c100610 0x7fb74c106ed0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 --2- 172.21.15.82:0/4122818588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb74c0fea90 0x7fb74c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 --2- 172.21.15.82:0/4122818588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74c0ffc80 0x7fb74c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 -- 172.21.15.82:0/4122818588 >> 172.21.15.82:0/4122818588 conn(0x7fb74c0fa090 msgr2=0x7fb74c0fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:31.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 -- 172.21.15.82:0/4122818588 shutdown_connections 2023-07-14T19:26:31.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.200+0000 7fb7528f4700 1 -- 172.21.15.82:0/4122818588 wait complete. 2023-07-14T19:26:31.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 Processor -- start 2023-07-14T19:26:31.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 -- start start 2023-07-14T19:26:31.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb74c0fea90 0x7fb74c068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:31.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 0x7fb74c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:31.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74c100610 0x7fb74c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c19d4c0 con 0x7fb74c0ffc80 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c19d630 con 0x7fb74c100610 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb7528f4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c19d7f0 con 0x7fb74c0fea90 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 0x7fb74c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 0x7fb74c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40458/0 (socket says 172.21.15.82:40458) 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb74b7fe700 1 -- 172.21.15.82:0/3197155885 learned_addr learned my addr 172.21.15.82:0/3197155885 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:31.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb74bfff700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb74c0fea90 0x7fb74c068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.201+0000 7fb750e91700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74c100610 0x7fb74c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb74b7fe700 1 -- 172.21.15.82:0/3197155885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb74c0fea90 msgr2=0x7fb74c068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb74b7fe700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb74c0fea90 0x7fb74c068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb74b7fe700 1 -- 172.21.15.82:0/3197155885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74c100610 msgr2=0x7fb74c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb74b7fe700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74c100610 0x7fb74c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb74b7fe700 1 -- 172.21.15.82:0/3197155885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744009d00 con 0x7fb74c0ffc80 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb74b7fe700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 0x7fb74c19a6e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00d950 tx=0x7fb73c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:31.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00f890 con 0x7fb74c0ffc80 2023-07-14T19:26:31.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb73c010460 con 0x7fb74c0ffc80 2023-07-14T19:26:31.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c19dad0 con 0x7fb74c0ffc80 2023-07-14T19:26:31.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.202+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c00e610 con 0x7fb74c0ffc80 2023-07-14T19:26:31.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.203+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c19dfe0 con 0x7fb74c0ffc80 2023-07-14T19:26:31.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.204+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c105f80 con 0x7fb74c0ffc80 2023-07-14T19:26:31.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.205+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb73c0108d0 con 0x7fb74c0ffc80 2023-07-14T19:26:31.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.205+0000 7fb7497fa700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb7380a11a0 0x7fb7380a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:31.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.205+0000 7fb74bfff700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb7380a11a0 0x7fb7380a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:31.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.205+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb73c0cc190 con 0x7fb74c0ffc80 2023-07-14T19:26:31.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.206+0000 7fb74bfff700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb7380a11a0 0x7fb7380a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0ffae0 tx=0x7fb734007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:31.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.209+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb73c095d80 con 0x7fb74c0ffc80 2023-07-14T19:26:31.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:30 smithi161 ceph-mon[129184]: Deploying daemon mds.j on smithi082 2023-07-14T19:26:31.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:30 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6832/1264023541,v1:172.21.15.133:6833/1264023541] up:boot 2023-07-14T19:26:31.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:30 smithi161 ceph-mon[129184]: fsmap 2 up:standby 2023-07-14T19:26:31.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:30 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T19:26:31.373 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:26:31 smithi082 podman[160635]: 2023-07-14T19:26:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.459+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb74c02cd70 con 0x7fb74c0ffc80 2023-07-14T19:26:31.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.460+0000 7fb7497fa700 1 -- 172.21.15.82:0/3197155885 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7fb73c0993a0 con 0x7fb74c0ffc80 2023-07-14T19:26:31.462 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:31.462 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-14T19:26:31.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb7380a11a0 msgr2=0x7fb7380a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:31.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb7380a11a0 0x7fb7380a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0ffae0 tx=0x7fb734007400 comp rx=0 tx=0).stop 2023-07-14T19:26:31.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 msgr2=0x7fb74c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:31.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 0x7fb74c19a6e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00d950 tx=0x7fb73c00dc60 comp rx=0 tx=0).stop 2023-07-14T19:26:31.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 shutdown_connections 2023-07-14T19:26:31.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74c100610 0x7fb74c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb74c0fea90 0x7fb74c068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb7380a11a0 0x7fb7380a3650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 --2- 172.21.15.82:0/3197155885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb74c0ffc80 0x7fb74c19a6e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:31.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 >> 172.21.15.82:0/3197155885 conn(0x7fb74c0fa090 msgr2=0x7fb74c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:31.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 shutdown_connections 2023-07-14T19:26:31.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:31.462+0000 7fb7528f4700 1 -- 172.21.15.82:0/3197155885 wait complete. 2023-07-14T19:26:31.465 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2023-07-14T19:26:31.715 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:26:31 smithi082 bash[160635]: fdd6b90b4c21b160ba0a59de08eefd04ac0248a8be0a3f2964191de31e242eff 2023-07-14T19:26:31.715 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mds[160718]: -- 172.21.15.82:0/2238868150 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9336eb1e0 con 0x55a9336c1000 2023-07-14T19:26:31.715 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-j[160675]: starting mds.j at 2023-07-14T19:26:31.715 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:26:31 smithi082 systemd[1]: Started Ceph mds.j for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3197155885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.115 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:32.116 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:32.116 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:31 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:32.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:32.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3197155885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:32.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:31 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:32.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3197155885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:32.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:31 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:33.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:32 smithi133 ceph-mon[128894]: Deploying daemon mds.d on smithi082 2023-07-14T19:26:33.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:32 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.82:6834/1071311199,v1:172.21.15.82:6835/1071311199] up:boot 2023-07-14T19:26:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:32 smithi133 ceph-mon[128894]: fsmap 3 up:standby 2023-07-14T19:26:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:32 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T19:26:33.214 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:26:32 smithi082 systemd[1]: Starting Ceph mds.d for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:32 smithi082 ceph-mon[127461]: Deploying daemon mds.d on smithi082 2023-07-14T19:26:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:32 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.82:6834/1071311199,v1:172.21.15.82:6835/1071311199] up:boot 2023-07-14T19:26:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:32 smithi082 ceph-mon[127461]: fsmap 3 up:standby 2023-07-14T19:26:33.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:32 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T19:26:33.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:32 smithi161 ceph-mon[129184]: Deploying daemon mds.d on smithi082 2023-07-14T19:26:33.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:32 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.82:6834/1071311199,v1:172.21.15.82:6835/1071311199] up:boot 2023-07-14T19:26:33.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:32 smithi161 ceph-mon[129184]: fsmap 3 up:standby 2023-07-14T19:26:33.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:32 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T19:26:33.478 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:33.565 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:26:33 smithi082 podman[161019]: 2023-07-14T19:26:33.566 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:26:33 smithi082 bash[161019]: 0c96277daa5eab467b3852236e9f5ee86099944ab315d366c5718591541a8e88 2023-07-14T19:26:33.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 -- 172.21.15.82:0/152547448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc100680 msgr2=0x7f1ecc106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/152547448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc100680 0x7f1ecc106f40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400b0a0 tx=0x7f1ec400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 -- 172.21.15.82:0/152547448 shutdown_connections 2023-07-14T19:26:33.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/152547448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc100680 0x7f1ecc106f40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/152547448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc0feb00 0x7f1ecc0fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/152547448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 -- 172.21.15.82:0/152547448 >> 172.21.15.82:0/152547448 conn(0x7f1ecc0fa100 msgr2=0x7f1ecc0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 -- 172.21.15.82:0/152547448 shutdown_connections 2023-07-14T19:26:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.617+0000 7f1ed3d17700 1 -- 172.21.15.82:0/152547448 wait complete. 2023-07-14T19:26:33.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 Processor -- start 2023-07-14T19:26:33.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 -- start start 2023-07-14T19:26:33.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ecc0feb00 0x7f1ecc068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:33.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc100680 0x7f1ecc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc19d490 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc19d600 con 0x7f1ecc0feb00 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed3d17700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc19d7c0 con 0x7f1ecc100680 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40498/0 (socket says 172.21.15.82:40498) 2023-07-14T19:26:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 -- 172.21.15.82:0/85935547 learned_addr learned my addr 172.21.15.82:0/85935547 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed1ab3700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ecc0feb00 0x7f1ecc068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed22b4700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc100680 0x7f1ecc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 -- 172.21.15.82:0/85935547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc100680 msgr2=0x7f1ecc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc100680 0x7f1ecc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 -- 172.21.15.82:0/85935547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ecc0feb00 msgr2=0x7f1ecc068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ecc0feb00 0x7f1ecc068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.618+0000 7f1ed12b2700 1 -- 172.21.15.82:0/85935547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec4009d00 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.619+0000 7f1ed12b2700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc19a660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c9b0 tx=0x7f1ebc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.619+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc007b00 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.619+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ebc007c60 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.619+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc018660 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.619+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc19daa0 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.619+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc19df60 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.620+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc1058c0 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.621+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ebc00f450 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.622+0000 7f1ec2ffd700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1eb80a11a0 0x7f1eb80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.622+0000 7f1ed1ab3700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1eb80a11a0 0x7f1eb80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.622+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ebc0cc700 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.622+0000 7f1ed1ab3700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1eb80a11a0 0x7f1eb80a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0ffb50 tx=0x7f1ec800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:33.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.625+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1ebc096380 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.856 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mds[161082]: -- 172.21.15.82:0/566069410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563311a711e0 con 0x563311a47000 2023-07-14T19:26:33.856 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-d[161035]: starting mds.d at 2023-07-14T19:26:33.856 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:26:33 smithi082 systemd[1]: Started Ceph mds.d for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.865+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ecc02c8e0 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.867 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.865+0000 7f1ec2ffd700 1 -- 172.21.15.82:0/85935547 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7f1ebc0999a0 con 0x7f1ecc0ffcf0 2023-07-14T19:26:33.867 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:33.867 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-14T19:26:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1eb80a11a0 msgr2=0x7f1eb80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1eb80a11a0 0x7f1eb80a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0ffb50 tx=0x7f1ec800b410 comp rx=0 tx=0).stop 2023-07-14T19:26:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 msgr2=0x7f1ecc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc19a660 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c9b0 tx=0x7f1ebc00ccc0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 shutdown_connections 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ecc100680 0x7f1ecc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ecc0feb00 0x7f1ecc068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1eb80a11a0 0x7f1eb80a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 --2- 172.21.15.82:0/85935547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ecc0ffcf0 0x7f1ecc19a660 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 >> 172.21.15.82:0/85935547 conn(0x7f1ecc0fa100 msgr2=0x7f1ecc0fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 shutdown_connections 2023-07-14T19:26:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:33.867+0000 7f1ed3d17700 1 -- 172.21.15.82:0/85935547 wait complete. 2023-07-14T19:26:33.871 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 4 2023-07-14T19:26:34.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:34.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:33 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:34.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:34.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:34.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:34.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:33 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:34.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:34.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:34.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:34.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:34.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:33 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:35.124 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:26:35 smithi133 systemd[1]: Starting Ceph mds.e for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:35.125 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:34 smithi133 ceph-mon[128894]: Deploying daemon mds.e on smithi133 2023-07-14T19:26:35.125 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:34 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/85935547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:35.125 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:34 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.82:6836/464864339,v1:172.21.15.82:6837/464864339] up:boot 2023-07-14T19:26:35.125 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:34 smithi133 ceph-mon[128894]: fsmap 4 up:standby 2023-07-14T19:26:35.125 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:34 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T19:26:35.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:34 smithi082 ceph-mon[127461]: Deploying daemon mds.e on smithi133 2023-07-14T19:26:35.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:34 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/85935547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:35.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:34 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.82:6836/464864339,v1:172.21.15.82:6837/464864339] up:boot 2023-07-14T19:26:35.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:34 smithi082 ceph-mon[127461]: fsmap 4 up:standby 2023-07-14T19:26:35.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:34 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T19:26:35.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:34 smithi161 ceph-mon[129184]: Deploying daemon mds.e on smithi133 2023-07-14T19:26:35.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:34 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/85935547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:35.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:34 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.82:6836/464864339,v1:172.21.15.82:6837/464864339] up:boot 2023-07-14T19:26:35.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:34 smithi161 ceph-mon[129184]: fsmap 4 up:standby 2023-07-14T19:26:35.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:34 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T19:26:35.684 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:26:35 smithi133 podman[153408]: 2023-07-14T19:26:35.684 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:26:35 smithi133 bash[153408]: c640fe9a4b77478b54ce2b47d88ed3e8a677a12afcc79fa781b39cc2aac37414 2023-07-14T19:26:35.684 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mds[153448]: -- 172.21.15.133:0/2275340298 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632804091e0 con 0x5632803de800 2023-07-14T19:26:35.685 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e[153424]: starting mds.e at 2023-07-14T19:26:35.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:36.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 -- 172.21.15.82:0/2996833859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240ff470 msgr2=0x7ff5240ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:36.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 --2- 172.21.15.82:0/2996833859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240ff470 0x7ff5240ff8c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009990 tx=0x7ff50c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 -- 172.21.15.82:0/2996833859 shutdown_connections 2023-07-14T19:26:36.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 --2- 172.21.15.82:0/2996833859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ffe00 0x7ff5241066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 --2- 172.21.15.82:0/2996833859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5240fe280 0x7ff5240fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 --2- 172.21.15.82:0/2996833859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240ff470 0x7ff5240ff8c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 -- 172.21.15.82:0/2996833859 >> 172.21.15.82:0/2996833859 conn(0x7ff5240f98a0 msgr2=0x7ff5240fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:36.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 -- 172.21.15.82:0/2996833859 shutdown_connections 2023-07-14T19:26:36.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 -- 172.21.15.82:0/2996833859 wait complete. 2023-07-14T19:26:36.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.017+0000 7ff529f2d700 1 Processor -- start 2023-07-14T19:26:36.020 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 -- start start 2023-07-14T19:26:36.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 0x7ff524194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:36.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ff470 0x7ff524194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:36.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5240ffe00 0x7ff52419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:36.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52419c3b0 con 0x7ff5240fe280 2023-07-14T19:26:36.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52419c4f0 con 0x7ff5240ffe00 2023-07-14T19:26:36.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52419c680 con 0x7ff5240ff470 2023-07-14T19:26:36.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 0x7ff524194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:36.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 0x7ff524194830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40512/0 (socket says 172.21.15.82:40512) 2023-07-14T19:26:36.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 -- 172.21.15.82:0/4245888250 learned_addr learned my addr 172.21.15.82:0/4245888250 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:36.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff523fff700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5240ffe00 0x7ff52419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:36.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff522ffd700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ff470 0x7ff524194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:36.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 -- 172.21.15.82:0/4245888250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ff470 msgr2=0x7ff524194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:36.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ff470 0x7ff524194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 -- 172.21.15.82:0/4245888250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5240ffe00 msgr2=0x7ff52419bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5240ffe00 0x7ff52419bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 -- 172.21.15.82:0/4245888250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0095f0 con 0x7ff5240fe280 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff522ffd700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ff470 0x7ff524194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff5237fe700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 0x7ff524194830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e970 tx=0x7ff51400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514009790 con 0x7ff5240fe280 2023-07-14T19:26:36.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52419c960 con 0x7ff5240fe280 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51400eec0 con 0x7ff5240fe280 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514003cf0 con 0x7ff5240fe280 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.018+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52419ce70 con 0x7ff5240fe280 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.019+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff524105770 con 0x7ff5240fe280 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.020+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff51400ccd0 con 0x7ff5240fe280 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.021+0000 7ff520ff9700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff5100a11a0 0x7ff5100a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:36.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.021+0000 7ff522ffd700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff5100a11a0 0x7ff5100a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:36.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.021+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff514014070 con 0x7ff5240fe280 2023-07-14T19:26:36.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.021+0000 7ff522ffd700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff5100a11a0 0x7ff5100a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00b2e0 tx=0x7ff50c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:36.027 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.024+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff5140963f0 con 0x7ff5240fe280 2023-07-14T19:26:36.183 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:26:35 smithi133 systemd[1]: Started Ceph mds.e for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:36.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:36.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:35 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:36.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:36.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:36.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:36.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:35 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:36.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:36.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:36.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:36.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:36.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:35 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:36.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.256+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff52419caf0 con 0x7ff5240fe280 2023-07-14T19:26:36.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.257+0000 7ff520ff9700 1 -- 172.21.15.82:0/4245888250 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3531 (secure 0 0 0) 0x7ff52419caf0 con 0x7ff5240fe280 2023-07-14T19:26:36.259 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:36.259 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-14T19:26:36.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff5100a11a0 msgr2=0x7ff5100a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:36.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff5100a11a0 0x7ff5100a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00b2e0 tx=0x7ff50c000bc0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 msgr2=0x7ff524194830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:36.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 0x7ff524194830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff51400e970 tx=0x7ff51400ec80 comp rx=0 tx=0).stop 2023-07-14T19:26:36.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 shutdown_connections 2023-07-14T19:26:36.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5240ffe00 0x7ff52419bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff5100a11a0 0x7ff5100a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5240fe280 0x7ff524194830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 --2- 172.21.15.82:0/4245888250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5240ff470 0x7ff524194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:36.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 >> 172.21.15.82:0/4245888250 conn(0x7ff5240f98a0 msgr2=0x7ff5241028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:36.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.259+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 shutdown_connections 2023-07-14T19:26:36.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:36.260+0000 7ff529f2d700 1 -- 172.21.15.82:0/4245888250 wait complete. 2023-07-14T19:26:36.265 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 5 2023-07-14T19:26:36.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:36 smithi161 ceph-mon[129184]: Deploying daemon mds.f on smithi161 2023-07-14T19:26:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:36 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4245888250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:36 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6834/3278917095,v1:172.21.15.133:6835/3278917095] up:boot 2023-07-14T19:26:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:36 smithi161 ceph-mon[129184]: fsmap 5 up:standby 2023-07-14T19:26:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:36 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:26:37.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:36 smithi133 ceph-mon[128894]: Deploying daemon mds.f on smithi161 2023-07-14T19:26:37.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:36 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4245888250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:37.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:36 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6834/3278917095,v1:172.21.15.133:6835/3278917095] up:boot 2023-07-14T19:26:37.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:36 smithi133 ceph-mon[128894]: fsmap 5 up:standby 2023-07-14T19:26:37.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:36 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:26:37.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:36 smithi082 ceph-mon[127461]: Deploying daemon mds.f on smithi161 2023-07-14T19:26:37.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:36 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4245888250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:37.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:36 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6834/3278917095,v1:172.21.15.133:6835/3278917095] up:boot 2023-07-14T19:26:37.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:36 smithi082 ceph-mon[127461]: fsmap 5 up:standby 2023-07-14T19:26:37.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:36 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:26:37.491 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:26:37 smithi161 systemd[1]: Starting Ceph mds.f for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:37.828 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:26:37 smithi161 podman[154608]: 2023-07-14T19:26:38.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:37 smithi133 ceph-mon[128894]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:38.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:37 smithi082 ceph-mon[127461]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:38.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:37 smithi161 ceph-mon[129184]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:38.241 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:26:37 smithi161 ceph-mds[154648]: -- 172.21.15.161:0/2808350185 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c72e4911e0 con 0x55c72e467000 2023-07-14T19:26:38.242 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:26:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f[154624]: starting mds.f at 2023-07-14T19:26:38.242 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:26:37 smithi161 bash[154608]: 01792752b19647305c8a55d9b4c8019ea7c55e652bb940af7d2131efe9b3870a 2023-07-14T19:26:38.242 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:26:37 smithi161 systemd[1]: Started Ceph mds.f for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:38.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 -- 172.21.15.82:0/1909853441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c103020 msgr2=0x7f367c105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:38.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 --2- 172.21.15.82:0/1909853441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c103020 0x7f367c105400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3670009990 tx=0x7f3670009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 -- 172.21.15.82:0/1909853441 shutdown_connections 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 --2- 172.21.15.82:0/1909853441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f367c105940 0x7f367c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 --2- 172.21.15.82:0/1909853441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f367c100700 0x7f367c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 --2- 172.21.15.82:0/1909853441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c103020 0x7f367c105400 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 -- 172.21.15.82:0/1909853441 >> 172.21.15.82:0/1909853441 conn(0x7f367c0fa100 msgr2=0x7f367c0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.406+0000 7f3681d62700 1 -- 172.21.15.82:0/1909853441 shutdown_connections 2023-07-14T19:26:38.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 -- 172.21.15.82:0/1909853441 wait complete. 2023-07-14T19:26:38.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 Processor -- start 2023-07-14T19:26:38.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 -- start start 2023-07-14T19:26:38.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 0x7f367c195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f367c103020 0x7f367c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f367c105940 0x7f367c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c19d3a0 con 0x7f367c100700 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c19d4e0 con 0x7f367c103020 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.407+0000 7f3681d62700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c19d670 con 0x7f367c105940 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 0x7f367c195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:38.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 0x7f367c195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40526/0 (socket says 172.21.15.82:40526) 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 -- 172.21.15.82:0/54532374 learned_addr learned my addr 172.21.15.82:0/54532374 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367bfff700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f367c105940 0x7f367c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367affd700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f367c103020 0x7f367c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 -- 172.21.15.82:0/54532374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f367c105940 msgr2=0x7f367c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f367c105940 0x7f367c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 -- 172.21.15.82:0/54532374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f367c103020 msgr2=0x7f367c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:38.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f367c103020 0x7f367c195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 -- 172.21.15.82:0/54532374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36700095f0 con 0x7f367c100700 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f367b7fe700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 0x7f367c195820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f366c00e9a0 tx=0x7f366c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c009790 con 0x7f367c100700 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f366c010430 con 0x7f367c100700 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c018680 con 0x7f367c100700 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c19d950 con 0x7f367c100700 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.408+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c19de10 con 0x7f367c100700 2023-07-14T19:26:38.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.409+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c0fbc70 con 0x7f367c100700 2023-07-14T19:26:38.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.411+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f366c0105a0 con 0x7f367c100700 2023-07-14T19:26:38.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.411+0000 7f3678ff9700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f36640a11a0 0x7f36640a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:38.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.411+0000 7f367affd700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f36640a11a0 0x7f36640a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:38.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.412+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f366c014070 con 0x7f367c100700 2023-07-14T19:26:38.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.412+0000 7f367affd700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f36640a11a0 0x7f36640a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f367000b2e0 tx=0x7f3670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:38.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.415+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f366c096410 con 0x7f367c100700 2023-07-14T19:26:38.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.649+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f367c02c8e0 con 0x7f367c100700 2023-07-14T19:26:38.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.650+0000 7f3678ff9700 1 -- 172.21.15.82:0/54532374 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5051 (secure 0 0 0) 0x7f366c099a30 con 0x7f367c100700 2023-07-14T19:26:38.651 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:38.651 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-14T19:26:38.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f36640a11a0 msgr2=0x7f36640a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:38.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f36640a11a0 0x7f36640a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f367000b2e0 tx=0x7f3670005fb0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 msgr2=0x7f367c195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 0x7f367c195820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f366c00e9a0 tx=0x7f366c00ecb0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 shutdown_connections 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f367c105940 0x7f367c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f36640a11a0 0x7f36640a3650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f367c100700 0x7f367c195820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 --2- 172.21.15.82:0/54532374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f367c103020 0x7f367c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:38.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.651+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 >> 172.21.15.82:0/54532374 conn(0x7f367c0fa100 msgr2=0x7f367c101360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:38.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.652+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 shutdown_connections 2023-07-14T19:26:38.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:38.652+0000 7f3681d62700 1 -- 172.21.15.82:0/54532374 wait complete. 2023-07-14T19:26:38.658 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 7 2023-07-14T19:26:39.383 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.384 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.384 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.384 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:39.384 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:39.384 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:39.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: Deploying daemon mds.c on smithi161 2023-07-14T19:26:39.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6834/3902735752,v1:172.21.15.161:6835/3902735752] up:boot 2023-07-14T19:26:39.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: fsmap 6 up:standby 2023-07-14T19:26:39.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:26:39.385 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:38 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/54532374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:39.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:39.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:39.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:38 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:39.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:39 smithi133 ceph-mon[128894]: Deploying daemon mds.c on smithi161 2023-07-14T19:26:39.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:39 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6834/3902735752,v1:172.21.15.161:6835/3902735752] up:boot 2023-07-14T19:26:39.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:39 smithi133 ceph-mon[128894]: fsmap 6 up:standby 2023-07-14T19:26:39.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:39 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:26:39.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:39 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/54532374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:39.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: Deploying daemon mds.c on smithi161 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6834/3902735752,v1:172.21.15.161:6835/3902735752] up:boot 2023-07-14T19:26:39.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: fsmap 6 up:standby 2023-07-14T19:26:39.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:26:39.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:38 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/54532374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:39.742 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:26:39 smithi161 systemd[1]: Starting Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:40.232 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:39 smithi161 ceph-mon[129184]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:40.233 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:26:39 smithi161 podman[154945]: 2023-07-14T19:26:40.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:39 smithi133 ceph-mon[128894]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:40.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:39 smithi082 ceph-mon[127461]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:40.491 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:26:40 smithi161 ceph-mds[154985]: -- 172.21.15.161:0/446035925 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623399c71e0 con 0x56233999c800 2023-07-14T19:26:40.492 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:26:40 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: starting mds.c at 2023-07-14T19:26:40.492 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:26:40 smithi161 bash[154945]: b78b76947e7c43b8aa38f720d6f06c6fa1f35f389e1d261c125e87284b31c3aa 2023-07-14T19:26:40.493 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:26:40 smithi161 systemd[1]: Started Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:40.666 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:40.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 -- 172.21.15.82:0/1498986380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 msgr2=0x7f98bc0ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:40.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 --2- 172.21.15.82:0/1498986380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0ff170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009990 tx=0x7f98b8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:40.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 -- 172.21.15.82:0/1498986380 shutdown_connections 2023-07-14T19:26:40.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 --2- 172.21.15.82:0/1498986380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc0ff6b0 0x7f98bc106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:40.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 --2- 172.21.15.82:0/1498986380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc1000b0 0x7f98bc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:40.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 --2- 172.21.15.82:0/1498986380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0ff170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:40.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 -- 172.21.15.82:0/1498986380 >> 172.21.15.82:0/1498986380 conn(0x7f98bc0f98a0 msgr2=0x7f98bc0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:40.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 -- 172.21.15.82:0/1498986380 shutdown_connections 2023-07-14T19:26:40.799 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.796+0000 7f98c457d700 1 -- 172.21.15.82:0/1498986380 wait complete. 2023-07-14T19:26:40.802 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 Processor -- start 2023-07-14T19:26:40.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 -- start start 2023-07-14T19:26:40.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0fe780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:40.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc0ff6b0 0x7f98bc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:40.803 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc1000b0 0x7f98bc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:40.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc19d430 con 0x7f98bc0fed00 2023-07-14T19:26:40.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc19d5a0 con 0x7f98bc1000b0 2023-07-14T19:26:40.804 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c457d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc19d760 con 0x7f98bc0ff6b0 2023-07-14T19:26:40.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0fe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0fe780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40540/0 (socket says 172.21.15.82:40540) 2023-07-14T19:26:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 -- 172.21.15.82:0/3716496410 learned_addr learned my addr 172.21.15.82:0/3716496410 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2b1a700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc1000b0 0x7f98bc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c1b18700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc0ff6b0 0x7f98bc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 -- 172.21.15.82:0/3716496410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc0ff6b0 msgr2=0x7f98bc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:40.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc0ff6b0 0x7f98bc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 -- 172.21.15.82:0/3716496410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc1000b0 msgr2=0x7f98bc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc1000b0 0x7f98bc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 -- 172.21.15.82:0/3716496410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80095f0 con 0x7f98bc0fed00 2023-07-14T19:26:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2b1a700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc1000b0 0x7f98bc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:40.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98c2319700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0fe780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d980 tx=0x7f98ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00bb60 con 0x7f98bc0fed00 2023-07-14T19:26:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98ac007640 con 0x7f98bc0fed00 2023-07-14T19:26:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.797+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004e90 con 0x7f98bc0fed00 2023-07-14T19:26:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.798+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc19da40 con 0x7f98bc0fed00 2023-07-14T19:26:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.798+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc19df00 con 0x7f98bc0fed00 2023-07-14T19:26:40.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.798+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc106100 con 0x7f98bc0fed00 2023-07-14T19:26:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.800+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f98ac00bcc0 con 0x7f98bc0fed00 2023-07-14T19:26:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.800+0000 7f98b37fe700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f98a80a1220 0x7f98a80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.800+0000 7f98c1b18700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f98a80a1220 0x7f98a80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.800+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f98ac0cc770 con 0x7f98bc0fed00 2023-07-14T19:26:40.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.800+0000 7f98c1b18700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f98a80a1220 0x7f98a80a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006010 tx=0x7f98b800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:40.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:40.803+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f98ac099df0 con 0x7f98bc0fed00 2023-07-14T19:26:41.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.035+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98bc02c8e0 con 0x7f98bc0fed00 2023-07-14T19:26:41.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.035+0000 7f98b37fe700 1 -- 172.21.15.82:0/3716496410 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5805 (secure 0 0 0) 0x7f98ac00de60 con 0x7f98bc0fed00 2023-07-14T19:26:41.037 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:41.037 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-14T19:26:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f98a80a1220 msgr2=0x7f98a80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f98a80a1220 0x7f98a80a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f98b8006010 tx=0x7f98b800b260 comp rx=0 tx=0).stop 2023-07-14T19:26:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 msgr2=0x7f98bc0fe780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0fe780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00d980 tx=0x7f98ac00dd40 comp rx=0 tx=0).stop 2023-07-14T19:26:41.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 shutdown_connections 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98bc1000b0 0x7f98bc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f98a80a1220 0x7f98a80a36d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98bc0fed00 0x7f98bc0fe780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 --2- 172.21.15.82:0/3716496410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98bc0ff6b0 0x7f98bc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 >> 172.21.15.82:0/3716496410 conn(0x7f98bc0f98a0 msgr2=0x7f98bc103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.037+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 shutdown_connections 2023-07-14T19:26:41.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:41.038+0000 7f98c457d700 1 -- 172.21.15.82:0/3716496410 wait complete. 2023-07-14T19:26:41.041 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2023-07-14T19:26:41.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.538 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:41.539 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:41.539 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:41.539 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: Deploying daemon mds.h on smithi133 2023-07-14T19:26:41.539 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:41.539 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6836/42384414,v1:172.21.15.161:6837/42384414] up:boot 2023-07-14T19:26:41.540 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: fsmap 7 up:standby 2023-07-14T19:26:41.540 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:26:41.540 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:41 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3716496410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:41.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:41.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:41.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:41.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: Deploying daemon mds.h on smithi133 2023-07-14T19:26:41.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:41.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6836/42384414,v1:172.21.15.161:6837/42384414] up:boot 2023-07-14T19:26:41.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: fsmap 7 up:standby 2023-07-14T19:26:41.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:26:41.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:41 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3716496410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:41.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:41.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:41.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:41.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:41.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: Deploying daemon mds.h on smithi133 2023-07-14T19:26:41.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:41.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6836/42384414,v1:172.21.15.161:6837/42384414] up:boot 2023-07-14T19:26:41.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: fsmap 7 up:standby 2023-07-14T19:26:41.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:26:41.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:41 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3716496410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:41.934 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:26:41 smithi133 systemd[1]: Starting Ceph mds.h for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:42.319 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:26:42 smithi133 podman[153750]: 2023-07-14T19:26:42.683 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:26:42 smithi133 ceph-mds[153790]: -- 172.21.15.133:0/53740594 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b97c7731e0 con 0x55b97c749000 2023-07-14T19:26:42.684 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:26:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: starting mds.h at 2023-07-14T19:26:42.684 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:26:42 smithi133 bash[153750]: e6a5bcc4fb4ac5839a22ead0f46b254f5cb192102237f49c9cee374f07a21fe9 2023-07-14T19:26:42.684 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:26:42 smithi133 systemd[1]: Started Ceph mds.h for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:43.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:43.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 -- 172.21.15.82:0/3360764754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ffe30 msgr2=0x7ff7d01066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:43.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 --2- 172.21.15.82:0/3360764754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ffe30 0x7ff7d01066f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800b0a0 tx=0x7ff7c800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 -- 172.21.15.82:0/3360764754 shutdown_connections 2023-07-14T19:26:43.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 --2- 172.21.15.82:0/3360764754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ffe30 0x7ff7d01066f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 --2- 172.21.15.82:0/3360764754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7d00fe2b0 0x7ff7d00fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 --2- 172.21.15.82:0/3360764754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d00ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 -- 172.21.15.82:0/3360764754 >> 172.21.15.82:0/3360764754 conn(0x7ff7d00f98b0 msgr2=0x7ff7d00fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 -- 172.21.15.82:0/3360764754 shutdown_connections 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 -- 172.21.15.82:0/3360764754 wait complete. 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 Processor -- start 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.184+0000 7ff7d7337700 1 -- start start 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7d00fe2b0 0x7ff7d00fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:43.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ffe30 0x7ff7d019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019d5c0 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019d730 con 0x7ff7d00ffe30 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d019d8f0 con 0x7ff7d00fe2b0 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40556/0 (socket says 172.21.15.82:40556) 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 -- 172.21.15.82:0/1552714456 learned_addr learned my addr 172.21.15.82:0/1552714456 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:43.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d50d3700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7d00fe2b0 0x7ff7d00fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:43.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d58d4700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ffe30 0x7ff7d019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:43.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 -- 172.21.15.82:0/1552714456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7d00fe2b0 msgr2=0x7ff7d00fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:43.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7d00fe2b0 0x7ff7d00fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 -- 172.21.15.82:0/1552714456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ffe30 msgr2=0x7ff7d019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ffe30 0x7ff7d019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 -- 172.21.15.82:0/1552714456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c8009d00 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d58d4700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ffe30 0x7ff7d019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d48d2700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d019a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00d950 tx=0x7ff7cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc009940 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7cc010460 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d019dbd0 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00f620 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.185+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d019e130 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.186+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d0105e20 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.188+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff7cc0108d0 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.189+0000 7ff7c67fc700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff7bc0a1220 0x7ff7bc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.189+0000 7ff7d50d3700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff7bc0a1220 0x7ff7bc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.189+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7cc0cc190 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.189+0000 7ff7d50d3700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff7bc0a1220 0x7ff7bc0a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00ff300 tx=0x7ff7c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:43.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.192+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff7cc095d80 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.425+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7d002c8e0 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.426+0000 7ff7c67fc700 1 -- 172.21.15.82:0/1552714456 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6559 (secure 0 0 0) 0x7ff7cc0993a0 con 0x7ff7d00ff4a0 2023-07-14T19:26:43.428 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:43.428 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-14T19:26:43.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.427+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff7bc0a1220 msgr2=0x7ff7bc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:43.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.427+0000 7ff7d7337700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff7bc0a1220 0x7ff7bc0a36d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00ff300 tx=0x7ff7c0009260 comp rx=0 tx=0).stop 2023-07-14T19:26:43.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.427+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 msgr2=0x7ff7d019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:43.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d019a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00d950 tx=0x7ff7cc00dc60 comp rx=0 tx=0).stop 2023-07-14T19:26:43.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 shutdown_connections 2023-07-14T19:26:43.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d00ffe30 0x7ff7d019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7d00fe2b0 0x7ff7d00fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff7bc0a1220 0x7ff7bc0a36d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 --2- 172.21.15.82:0/1552714456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7d00ff4a0 0x7ff7d019a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:43.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 >> 172.21.15.82:0/1552714456 conn(0x7ff7d00f98b0 msgr2=0x7ff7d0102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:43.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 shutdown_connections 2023-07-14T19:26:43.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:43.428+0000 7ff7d7337700 1 -- 172.21.15.82:0/1552714456 wait complete. 2023-07-14T19:26:43.432 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 9 2023-07-14T19:26:43.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:boot 2023-07-14T19:26:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: fsmap 8 up:standby 2023-07-14T19:26:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:26:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:43.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:43.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:43.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: Deploying daemon mds.i on smithi161 2023-07-14T19:26:43.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:43 smithi133 ceph-mon[128894]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:43.690 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:boot 2023-07-14T19:26:43.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: fsmap 8 up:standby 2023-07-14T19:26:43.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:26:43.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.691 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:43.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:43.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:43.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: Deploying daemon mds.i on smithi161 2023-07-14T19:26:43.692 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:43 smithi161 ceph-mon[129184]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:43.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:boot 2023-07-14T19:26:43.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: fsmap 8 up:standby 2023-07-14T19:26:43.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:26:43.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:43.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:43.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:43.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:43.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: Deploying daemon mds.i on smithi161 2023-07-14T19:26:43.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:43 smithi082 ceph-mon[127461]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:43.991 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:26:43 smithi161 systemd[1]: Starting Ceph mds.i for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:44.474 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:44 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1552714456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:44.475 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:26:44 smithi161 podman[155283]: 2023-07-14T19:26:44.475 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:26:44 smithi161 ceph-mds[155323]: -- 172.21.15.161:0/2180741531 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d6061b1e0 con 0x555d605f0800 2023-07-14T19:26:44.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:44 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1552714456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:44.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:44 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1552714456' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:44.741 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:26:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-i[155299]: starting mds.i at 2023-07-14T19:26:44.742 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:26:44 smithi161 bash[155283]: b63e4500df0072cf705cfe0b41da4bfebf688f92bab8cc1b54e485beb1c23a3a 2023-07-14T19:26:44.742 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:26:44 smithi161 systemd[1]: Started Ceph mds.i for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:45.443 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 -- 172.21.15.82:0/2906963439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0ffcf0 msgr2=0x7fe9dc100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 --2- 172.21.15.82:0/2906963439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0ffcf0 0x7fe9dc100140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009990 tx=0x7fe9cc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 -- 172.21.15.82:0/2906963439 shutdown_connections 2023-07-14T19:26:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 --2- 172.21.15.82:0/2906963439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9dc100680 0x7fe9dc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 --2- 172.21.15.82:0/2906963439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc0fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 --2- 172.21.15.82:0/2906963439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0ffcf0 0x7fe9dc100140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 -- 172.21.15.82:0/2906963439 >> 172.21.15.82:0/2906963439 conn(0x7fe9dc0fa100 msgr2=0x7fe9dc0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:45.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 -- 172.21.15.82:0/2906963439 shutdown_connections 2023-07-14T19:26:45.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.581+0000 7fe9e2031700 1 -- 172.21.15.82:0/2906963439 wait complete. 2023-07-14T19:26:45.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 Processor -- start 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 -- start start 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9dc0ffcf0 0x7fe9dc194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc100680 0x7fe9dc19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc19c380 con 0x7fe9dc0feb00 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc19c4c0 con 0x7fe9dc0ffcf0 2023-07-14T19:26:45.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9e2031700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc19c650 con 0x7fe9dc100680 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40568/0 (socket says 172.21.15.82:40568) 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9db7fe700 1 -- 172.21.15.82:0/3581918665 learned_addr learned my addr 172.21.15.82:0/3581918665 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.582+0000 7fe9daffd700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9dc0ffcf0 0x7fe9dc194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9dbfff700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc100680 0x7fe9dc19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9db7fe700 1 -- 172.21.15.82:0/3581918665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc100680 msgr2=0x7fe9dc19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:45.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9db7fe700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc100680 0x7fe9dc19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9db7fe700 1 -- 172.21.15.82:0/3581918665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9dc0ffcf0 msgr2=0x7fe9dc194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9db7fe700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9dc0ffcf0 0x7fe9dc194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9db7fe700 1 -- 172.21.15.82:0/3581918665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0095f0 con 0x7fe9dc0feb00 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9db7fe700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc194800 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c970 tx=0x7fe9c400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4004220 con 0x7fe9dc0feb00 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc19c930 con 0x7fe9dc0feb00 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c4007d60 con 0x7fe9dc0feb00 2023-07-14T19:26:45.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4003c90 con 0x7fe9dc0feb00 2023-07-14T19:26:45.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.583+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc19cec0 con 0x7fe9dc0feb00 2023-07-14T19:26:45.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.584+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc106020 con 0x7fe9dc0feb00 2023-07-14T19:26:45.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.585+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe9c4004380 con 0x7fe9dc0feb00 2023-07-14T19:26:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.585+0000 7fe9d8ff9700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9c80a1220 0x7fe9c80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.586+0000 7fe9daffd700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9c80a1220 0x7fe9c80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.586+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe9c40cc460 con 0x7fe9dc0feb00 2023-07-14T19:26:45.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.586+0000 7fe9daffd700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9c80a1220 0x7fe9c80a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b5c0 tx=0x7fe9cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:45.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.589+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe9c4095fb0 con 0x7fe9dc0feb00 2023-07-14T19:26:45.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.828+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9dc02c8e0 con 0x7fe9dc0feb00 2023-07-14T19:26:45.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.829+0000 7fe9d8ff9700 1 -- 172.21.15.82:0/3581918665 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7321 (secure 0 0 0) 0x7fe9c40995d0 con 0x7fe9dc0feb00 2023-07-14T19:26:45.831 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:45.831 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-14T19:26:45.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.831 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:45.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:45.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:45.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: Deploying daemon mds.g on smithi082 2023-07-14T19:26:45.832 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:45.833 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] up:boot 2023-07-14T19:26:45.833 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: fsmap 9 up:standby 2023-07-14T19:26:45.833 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:45 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:26:45.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9c80a1220 msgr2=0x7fe9c80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:45.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9c80a1220 0x7fe9c80a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00b5c0 tx=0x7fe9cc005960 comp rx=0 tx=0).stop 2023-07-14T19:26:45.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 msgr2=0x7fe9dc194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc194800 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c970 tx=0x7fe9c400cc80 comp rx=0 tx=0).stop 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 shutdown_connections 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9dc100680 0x7fe9dc19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9c80a1220 0x7fe9c80a36d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9dc0feb00 0x7fe9dc194800 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 --2- 172.21.15.82:0/3581918665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9dc0ffcf0 0x7fe9dc194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 >> 172.21.15.82:0/3581918665 conn(0x7fe9dc0fa100 msgr2=0x7fe9dc103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:45.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.831+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 shutdown_connections 2023-07-14T19:26:45.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:45.832+0000 7fe9e2031700 1 -- 172.21.15.82:0/3581918665 wait complete. 2023-07-14T19:26:45.836 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 10 2023-07-14T19:26:45.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:45.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:45.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:45.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: Deploying daemon mds.g on smithi082 2023-07-14T19:26:45.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:45.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] up:boot 2023-07-14T19:26:45.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: fsmap 9 up:standby 2023-07-14T19:26:45.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:45 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:26:45.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:45.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:45.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:45.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:45.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: Deploying daemon mds.g on smithi082 2023-07-14T19:26:45.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:45.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] up:boot 2023-07-14T19:26:45.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: fsmap 9 up:standby 2023-07-14T19:26:45.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:45 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:26:46.214 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:26:46 smithi082 systemd[1]: Starting Ceph mds.g for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:46.587 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:26:46 smithi082 podman[161630]: 2023-07-14T19:26:46.588 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:26:46 smithi082 ceph-mds[161670]: -- 172.21.15.82:0/3978451203 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd300d51e0 con 0x55cd300aac00 2023-07-14T19:26:46.588 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:26:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-g[161646]: starting mds.g at 2023-07-14T19:26:46.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:46 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3581918665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:46.964 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:26:46 smithi082 bash[161630]: f704df9e1981b9b8c6be1261e3ab51482083b7232b7b85d37fc6647887d6c5c7 2023-07-14T19:26:46.964 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:26:46 smithi082 systemd[1]: Started Ceph mds.g for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:46.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:46 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3581918665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:46.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:46 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3581918665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:47.601 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.82:6838/3224571334,v1:172.21.15.82:6839/3224571334] up:boot 2023-07-14T19:26:47.601 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: fsmap 10 up:standby 2023-07-14T19:26:47.601 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T19:26:47.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:47.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:47.602 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:47.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:47.603 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:47 smithi082 ceph-mon[127461]: Deploying daemon mds.a on smithi082 2023-07-14T19:26:47.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:47.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.82:6838/3224571334,v1:172.21.15.82:6839/3224571334] up:boot 2023-07-14T19:26:47.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: fsmap 10 up:standby 2023-07-14T19:26:47.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T19:26:47.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:47.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:47.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:47.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:47.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:47 smithi133 ceph-mon[128894]: Deploying daemon mds.a on smithi082 2023-07-14T19:26:47.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.82:6838/3224571334,v1:172.21.15.82:6839/3224571334] up:boot 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: fsmap 10 up:standby 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:47.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:47.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:47.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:47.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:47 smithi161 ceph-mon[129184]: Deploying daemon mds.a on smithi082 2023-07-14T19:26:47.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.993+0000 7fcafb025700 1 -- 172.21.15.82:0/1606818173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf40fe630 msgr2=0x7fcaf4108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:47.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.993+0000 7fcafb025700 1 --2- 172.21.15.82:0/1606818173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf40fe630 0x7fcaf4108210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b0a0 tx=0x7fcaec00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:47.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 -- 172.21.15.82:0/1606818173 shutdown_connections 2023-07-14T19:26:47.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 --2- 172.21.15.82:0/1606818173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf40fe630 0x7fcaf4108210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:47.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 --2- 172.21.15.82:0/1606818173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf4101230 0x7fcaf4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:47.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 --2- 172.21.15.82:0/1606818173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaf40fdca0 0x7fcaf40fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:47.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 -- 172.21.15.82:0/1606818173 >> 172.21.15.82:0/1606818173 conn(0x7fcaf40f98d0 msgr2=0x7fcaf40fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:47.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 -- 172.21.15.82:0/1606818173 shutdown_connections 2023-07-14T19:26:47.996 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.994+0000 7fcafb025700 1 -- 172.21.15.82:0/1606818173 wait complete. 2023-07-14T19:26:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 Processor -- start 2023-07-14T19:26:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 -- start start 2023-07-14T19:26:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaf40fdca0 0x7fcaf4100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf40fe630 0x7fcaf40fef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 0x7fcaf40ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40ffc50 con 0x7fcaf4101230 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40ffdc0 con 0x7fcaf40fdca0 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcafb025700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf419fa70 con 0x7fcaf40fe630 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 0x7fcaf40ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 0x7fcaf40ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40596/0 (socket says 172.21.15.82:40596) 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 -- 172.21.15.82:0/3832974219 learned_addr learned my addr 172.21.15.82:0/3832974219 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf8dc1700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaf40fdca0 0x7fcaf4100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf3fff700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf40fe630 0x7fcaf40fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 -- 172.21.15.82:0/3832974219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf40fe630 msgr2=0x7fcaf40fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf40fe630 0x7fcaf40fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 -- 172.21.15.82:0/3832974219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaf40fdca0 msgr2=0x7fcaf4100930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaf40fdca0 0x7fcaf4100930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.995+0000 7fcaf95c2700 1 -- 172.21.15.82:0/3832974219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec009d00 con 0x7fcaf4101230 2023-07-14T19:26:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.996+0000 7fcaf95c2700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 0x7fcaf40ff4b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b070 tx=0x7fcaec004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.996+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00e040 con 0x7fcaf4101230 2023-07-14T19:26:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.996+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaec004020 con 0x7fcaf4101230 2023-07-14T19:26:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.996+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf419fcf0 con 0x7fcaf4101230 2023-07-14T19:26:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.996+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec01b6b0 con 0x7fcaf4101230 2023-07-14T19:26:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.996+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41a0170 con 0x7fcaf4101230 2023-07-14T19:26:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.997+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf410b490 con 0x7fcaf4101230 2023-07-14T19:26:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.998+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcaec004190 con 0x7fcaf4101230 2023-07-14T19:26:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.998+0000 7fcaf1ffb700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcae40a11a0 0x7fcae40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.999+0000 7fcaf8dc1700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcae40a11a0 0x7fcae40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.999+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcaec0cda50 con 0x7fcaf4101230 2023-07-14T19:26:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:47.999+0000 7fcaf8dc1700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcae40a11a0 0x7fcae40a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005fd0 tx=0x7fcae8005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.006+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fcaec0976d0 con 0x7fcaf4101230 2023-07-14T19:26:48.261 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.259+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf4106be0 con 0x7fcaf4101230 2023-07-14T19:26:48.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.260+0000 7fcaf1ffb700 1 -- 172.21.15.82:0/3832974219 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8079 (secure 0 0 0) 0x7fcaec09acf0 con 0x7fcaf4101230 2023-07-14T19:26:48.262 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:48.262 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-14T19:26:48.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcae40a11a0 msgr2=0x7fcae40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:48.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcae40a11a0 0x7fcae40a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005fd0 tx=0x7fcae8005dd0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 msgr2=0x7fcaf40ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 0x7fcaf40ff4b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b070 tx=0x7fcaec004b00 comp rx=0 tx=0).stop 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 shutdown_connections 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcae40a11a0 0x7fcae40a3650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaf4101230 0x7fcaf40ff4b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcaf40fdca0 0x7fcaf4100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 --2- 172.21.15.82:0/3832974219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcaf40fe630 0x7fcaf40fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:48.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 >> 172.21.15.82:0/3832974219 conn(0x7fcaf40f98d0 msgr2=0x7fcaf40fba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:48.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 shutdown_connections 2023-07-14T19:26:48.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:48.264+0000 7fcafb025700 1 -- 172.21.15.82:0/3832974219 wait complete. 2023-07-14T19:26:48.267 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 11 2023-07-14T19:26:48.464 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:26:48 smithi082 systemd[1]: Starting Ceph mds.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:48.744 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:26:48 smithi082 podman[162019]: 2023-07-14T19:26:48.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:48 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3832974219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:48.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:48 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3832974219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:49.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:48 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3832974219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:49.214 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:26:48 smithi082 ceph-mds[162059]: -- 172.21.15.82:0/1235419323 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653a760d1e0 con 0x5653a75e2800 2023-07-14T19:26:49.214 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:26:48 smithi082 bash[162019]: 7196b46d9cffbfd2d5cb1bd73f7e39adce39f100b3161163919dfaf75115f2c8 2023-07-14T19:26:49.215 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:26:48 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-a[162035]: starting mds.a at 2023-07-14T19:26:49.215 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:26:48 smithi082 systemd[1]: Started Ceph mds.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:50.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:50.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.135 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.136 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.136 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:50.136 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:50.136 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:50.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: Deploying daemon mds.b on smithi133 2023-07-14T19:26:50.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.82:6840/2495420363,v1:172.21.15.82:6841/2495420363] up:boot 2023-07-14T19:26:50.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: fsmap 11 up:standby 2023-07-14T19:26:50.137 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:49 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T19:26:50.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:50.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: Deploying daemon mds.b on smithi133 2023-07-14T19:26:50.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.82:6840/2495420363,v1:172.21.15.82:6841/2495420363] up:boot 2023-07-14T19:26:50.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: fsmap 11 up:standby 2023-07-14T19:26:50.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:49 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T19:26:50.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-14T19:26:50.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-14T19:26:50.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:50.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: Deploying daemon mds.b on smithi133 2023-07-14T19:26:50.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.82:6840/2495420363,v1:172.21.15.82:6841/2495420363] up:boot 2023-07-14T19:26:50.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: fsmap 11 up:standby 2023-07-14T19:26:50.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:49 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T19:26:50.282 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:50.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 -- 172.21.15.82:0/3336396562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 msgr2=0x7fc46c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:50.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 --2- 172.21.15.82:0/3336396562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c0fef20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc4680098f0 tx=0x7fc468009c00 comp rx=0 tx=0).stop 2023-07-14T19:26:50.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 -- 172.21.15.82:0/3336396562 shutdown_connections 2023-07-14T19:26:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 --2- 172.21.15.82:0/3336396562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc46c100690 0x7fc46c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 --2- 172.21.15.82:0/3336396562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c0fef20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 --2- 172.21.15.82:0/3336396562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c0ffd00 0x7fc46c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.421+0000 7fc473665700 1 -- 172.21.15.82:0/3336396562 >> 172.21.15.82:0/3336396562 conn(0x7fc46c0fa130 msgr2=0x7fc46c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 -- 172.21.15.82:0/3336396562 shutdown_connections 2023-07-14T19:26:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 -- 172.21.15.82:0/3336396562 wait complete. 2023-07-14T19:26:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 Processor -- start 2023-07-14T19:26:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 -- start start 2023-07-14T19:26:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc46c0ffd00 0x7fc46c195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100690 0x7fc46c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19d5e0 con 0x7fc46c0feb10 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19d720 con 0x7fc46c0ffd00 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc473665700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c19d8b0 con 0x7fc46c100690 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc471401700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc471401700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40620/0 (socket says 172.21.15.82:40620) 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc471401700 1 -- 172.21.15.82:0/2529713972 learned_addr learned my addr 172.21.15.82:0/2529713972 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.422+0000 7fc470c00700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc46c0ffd00 0x7fc46c195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471c02700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100690 0x7fc46c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471401700 1 -- 172.21.15.82:0/2529713972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100690 msgr2=0x7fc46c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471401700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100690 0x7fc46c19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471401700 1 -- 172.21.15.82:0/2529713972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc46c0ffd00 msgr2=0x7fc46c195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471401700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc46c0ffd00 0x7fc46c195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471401700 1 -- 172.21.15.82:0/2529713972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468009520 con 0x7fc46c0feb10 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471c02700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100690 0x7fc46c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:26:50.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc471401700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c195a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc468009fd0 tx=0x7fc46800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46801c070 con 0x7fc46c0feb10 2023-07-14T19:26:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc468005040 con 0x7fc46c0feb10 2023-07-14T19:26:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c19db30 con 0x7fc46c0feb10 2023-07-14T19:26:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468003a40 con 0x7fc46c0feb10 2023-07-14T19:26:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.423+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c19df50 con 0x7fc46c0feb10 2023-07-14T19:26:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.424+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c106050 con 0x7fc46c0feb10 2023-07-14T19:26:50.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.425+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc468020030 con 0x7fc46c0feb10 2023-07-14T19:26:50.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.426+0000 7fc4627fc700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4580a11a0 0x7fc4580a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:50.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.426+0000 7fc470c00700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4580a11a0 0x7fc4580a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:50.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.426+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4680cd9c0 con 0x7fc46c0feb10 2023-07-14T19:26:50.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.426+0000 7fc470c00700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4580a11a0 0x7fc4580a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005fd0 tx=0x7fc45c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:50.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.429+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc468097640 con 0x7fc46c0feb10 2023-07-14T19:26:50.434 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:26:50 smithi133 systemd[1]: Starting Ceph mds.b for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:26:50.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.662+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc46c02c8e0 con 0x7fc46c0feb10 2023-07-14T19:26:50.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.662+0000 7fc4627fc700 1 -- 172.21.15.82:0/2529713972 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8837 (secure 0 0 0) 0x7fc46809ac60 con 0x7fc46c0feb10 2023-07-14T19:26:50.664 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:50.664 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-14T19:26:50.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4580a11a0 msgr2=0x7fc4580a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:50.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4580a11a0 0x7fc4580a3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005fd0 tx=0x7fc45c009310 comp rx=0 tx=0).stop 2023-07-14T19:26:50.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 msgr2=0x7fc46c195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:50.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c195a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc468009fd0 tx=0x7fc46800b5b0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 shutdown_connections 2023-07-14T19:26:50.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc46c100690 0x7fc46c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc4580a11a0 0x7fc4580a3650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc46c0feb10 0x7fc46c195a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 --2- 172.21.15.82:0/2529713972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc46c0ffd00 0x7fc46c195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:50.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 >> 172.21.15.82:0/2529713972 conn(0x7fc46c0fa130 msgr2=0x7fc46c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:50.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 shutdown_connections 2023-07-14T19:26:50.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:50.664+0000 7fc473665700 1 -- 172.21.15.82:0/2529713972 wait complete. 2023-07-14T19:26:50.668 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 12 2023-07-14T19:26:50.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:50 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2529713972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:50.934 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:26:50 smithi133 podman[154089]: 2023-07-14T19:26:50.935 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:26:50 smithi133 bash[154089]: 40b3c59af0bb7cff536496154508b21179ad8596cc5f9e449eb6816ddcc89b69 2023-07-14T19:26:50.935 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:26:50 smithi133 ceph-mds[154129]: -- 172.21.15.133:0/1314076236 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5642e046d1e0 con 0x5642e0442800 2023-07-14T19:26:50.935 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:26:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: starting mds.b at 2023-07-14T19:26:50.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:50 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2529713972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:51.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:50 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2529713972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:51.433 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:26:50 smithi133 systemd[1]: Started Ceph mds.b for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:26:52.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:52.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] up:boot 2023-07-14T19:26:52.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: fsmap 12 up:standby 2023-07-14T19:26:52.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:52 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:26:52.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:52.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:52.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] up:boot 2023-07-14T19:26:52.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: fsmap 12 up:standby 2023-07-14T19:26:52.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:52 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:26:52.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:52.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:52.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:52.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] up:boot 2023-07-14T19:26:52.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: fsmap 12 up:standby 2023-07-14T19:26:52.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:52 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:26:52.680 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:26:52.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.805+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3324849194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 msgr2=0x7ff9f4106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:52.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.805+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3324849194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f4106f90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00b0a0 tx=0x7ff9ec00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3324849194 shutdown_connections 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3324849194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f4106f90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3324849194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f40fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3324849194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 0x7ff9f4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3324849194 >> 172.21.15.82:0/3324849194 conn(0x7ff9f40fa150 msgr2=0x7ff9f40fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3324849194 shutdown_connections 2023-07-14T19:26:52.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3324849194 wait complete. 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.806+0000 7ff9f88bb700 1 Processor -- start 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 -- start start 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f4194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 0x7ff9f4194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f419c4a0 con 0x7ff9f41006d0 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f419c5e0 con 0x7ff9f40ffd40 2023-07-14T19:26:52.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f88bb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f419c770 con 0x7ff9f40feb50 2023-07-14T19:26:52.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f4194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:52.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:52.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f259c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f4194920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:49898/0 (socket says 172.21.15.82:49898) 2023-07-14T19:26:52.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40638/0 (socket says 172.21.15.82:40638) 2023-07-14T19:26:52.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 0x7ff9f4194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:52.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f259c700 1 -- 172.21.15.82:0/3516117034 learned_addr learned my addr 172.21.15.82:0/3516117034 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:52.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 -- 172.21.15.82:0/3516117034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 msgr2=0x7ff9f4194920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:52.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f4194920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:52.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 -- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 msgr2=0x7ff9f419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:52.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:52.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 -- 172.21.15.82:0/3516117034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec009d00 con 0x7ff9f40ffd40 2023-07-14T19:26:52.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f259c700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f4194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:26:52.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f2d9d700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:26:52.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.807+0000 7ff9f1d9b700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 0x7ff9f4194e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800e9f0 tx=0x7ff9e800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:52.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.808+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800c9c0 con 0x7ff9f40ffd40 2023-07-14T19:26:52.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.808+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f419ca50 con 0x7ff9f40ffd40 2023-07-14T19:26:52.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.808+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e8010430 con 0x7ff9f40ffd40 2023-07-14T19:26:52.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.808+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8018680 con 0x7ff9f40ffd40 2023-07-14T19:26:52.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.808+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f419cf60 con 0x7ff9f40ffd40 2023-07-14T19:26:52.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.809+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f4105fa0 con 0x7ff9f40ffd40 2023-07-14T19:26:52.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.812+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff9e8010860 con 0x7ff9f40ffd40 2023-07-14T19:26:52.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.812+0000 7ff9e37fe700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff9dc0a11a0 0x7ff9dc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:52.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.812+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9e8014070 con 0x7ff9f40ffd40 2023-07-14T19:26:52.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.813+0000 7ff9f259c700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff9dc0a11a0 0x7ff9dc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:52.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.814+0000 7ff9f259c700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff9dc0a11a0 0x7ff9dc0a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4005950 tx=0x7ff9e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:52.816 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:52.815+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff9e8096300 con 0x7ff9f40ffd40 2023-07-14T19:26:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.059+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9f402c8e0 con 0x7ff9f40ffd40 2023-07-14T19:26:53.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.060+0000 7ff9e37fe700 1 -- 172.21.15.82:0/3516117034 <== mon.2 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9595 (secure 0 0 0) 0x7ff9e8099920 con 0x7ff9f40ffd40 2023-07-14T19:26:53.062 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:53.062 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24491,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-14T19:26:53.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff9dc0a11a0 msgr2=0x7ff9dc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff9dc0a11a0 0x7ff9dc0a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4005950 tx=0x7ff9e4009260 comp rx=0 tx=0).stop 2023-07-14T19:26:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 msgr2=0x7ff9f4194e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 0x7ff9f4194e60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800e9f0 tx=0x7ff9e800ed00 comp rx=0 tx=0).stop 2023-07-14T19:26:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 shutdown_connections 2023-07-14T19:26:53.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff9dc0a11a0 0x7ff9dc0a3650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f41006d0 0x7ff9f419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9f40feb50 0x7ff9f4194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 --2- 172.21.15.82:0/3516117034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f40ffd40 0x7ff9f4194e60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 >> 172.21.15.82:0/3516117034 conn(0x7ff9f40fa150 msgr2=0x7ff9f41030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:53.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 shutdown_connections 2023-07-14T19:26:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.063+0000 7ff9f88bb700 1 -- 172.21.15.82:0/3516117034 wait complete. 2023-07-14T19:26:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 13 2023-07-14T19:26:53.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:26:53.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 -- 172.21.15.82:0/1812226072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 msgr2=0x7fbc040fe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1812226072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc040fe760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 -- 172.21.15.82:0/1812226072 shutdown_connections 2023-07-14T19:26:53.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1812226072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ffed0 0x7fbc04106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1812226072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc040fe760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1812226072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc040ff540 0x7fbc040ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 -- 172.21.15.82:0/1812226072 >> 172.21.15.82:0/1812226072 conn(0x7fbc040f9990 msgr2=0x7fbc040fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 -- 172.21.15.82:0/1812226072 shutdown_connections 2023-07-14T19:26:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.201+0000 7fbc0989c700 1 -- 172.21.15.82:0/1812226072 wait complete. 2023-07-14T19:26:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 Processor -- start 2023-07-14T19:26:53.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 -- start start 2023-07-14T19:26:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc041022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ff540 0x7fbc04100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc040ffed0 0x7fbc04100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04101600 con 0x7fbc040fe350 2023-07-14T19:26:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04101770 con 0x7fbc040ff540 2023-07-14T19:26:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc0989c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0419f980 con 0x7fbc040ffed0 2023-07-14T19:26:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc02ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc041022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.202+0000 7fbc02ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc041022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40646/0 (socket says 172.21.15.82:40646) 2023-07-14T19:26:53.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 -- 172.21.15.82:0/1529311032 learned_addr learned my addr 172.21.15.82:0/1529311032 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:53.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc037fe700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc040ffed0 0x7fbc04100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc027fc700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ff540 0x7fbc04100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 -- 172.21.15.82:0/1529311032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc040ffed0 msgr2=0x7fbc04100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc040ffed0 0x7fbc04100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 -- 172.21.15.82:0/1529311032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ff540 msgr2=0x7fbc04100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ff540 0x7fbc04100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 -- 172.21.15.82:0/1529311032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40095f0 con 0x7fbc040fe350 2023-07-14T19:26:53.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc027fc700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ff540 0x7fbc04100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:26:53.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc02ffd700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc041022e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4000c00 tx=0x7fbbf400f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:53.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf401d070 con 0x7fbc040fe350 2023-07-14T19:26:53.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbf400fc60 con 0x7fbc040fe350 2023-07-14T19:26:53.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0419fc00 con 0x7fbc040fe350 2023-07-14T19:26:53.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4017650 con 0x7fbc040fe350 2023-07-14T19:26:53.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.203+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc041a0020 con 0x7fbc040fe350 2023-07-14T19:26:53.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.204+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc04106120 con 0x7fbc040fe350 2023-07-14T19:26:53.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.205+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbbf4004830 con 0x7fbc040fe350 2023-07-14T19:26:53.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.206+0000 7fbc0889a700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbbec0a11a0 0x7fbbec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.206+0000 7fbc027fc700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbbec0a11a0 0x7fbbec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.206+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbbf40cdb10 con 0x7fbc040fe350 2023-07-14T19:26:53.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.206+0000 7fbc027fc700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbbec0a11a0 0x7fbbec0a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8005fd0 tx=0x7fbbf8005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:53.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.209+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fbbf4097790 con 0x7fbc040fe350 2023-07-14T19:26:53.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.416+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc04109bb0 con 0x7fbc040fe350 2023-07-14T19:26:53.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.418+0000 7fbc0889a700 1 -- 172.21.15.82:0/1529311032 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18432 (secure 0 0 0) 0x7fbbf409adb0 con 0x7fbc040fe350 2023-07-14T19:26:53.419 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:53.419 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":77,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:26:17.073692+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:26:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbbec0a11a0 msgr2=0x7fbbec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbbec0a11a0 0x7fbbec0a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8005fd0 tx=0x7fbbf8005dc0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 msgr2=0x7fbc041022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc041022e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4000c00 tx=0x7fbbf400f510 comp rx=0 tx=0).stop 2023-07-14T19:26:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 shutdown_connections 2023-07-14T19:26:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc040ffed0 0x7fbc04100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbbec0a11a0 0x7fbbec0a3650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc040fe350 0x7fbc041022e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 --2- 172.21.15.82:0/1529311032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc040ff540 0x7fbc04100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 >> 172.21.15.82:0/1529311032 conn(0x7fbc040f9990 msgr2=0x7fbc0410ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 shutdown_connections 2023-07-14T19:26:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.420+0000 7fbc0989c700 1 -- 172.21.15.82:0/1529311032 wait complete. 2023-07-14T19:26:53.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:26:53.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:26:53.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/47926242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00fe9d0 msgr2=0x7f4cd00fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/47926242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd00fede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009990 tx=0x7f4cc0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/47926242 shutdown_connections 2023-07-14T19:26:53.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/47926242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd0100550 0x7f4cd0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/47926242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd00fede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/47926242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/47926242 >> 172.21.15.82:0/47926242 conn(0x7f4cd00f9fd0 msgr2=0x7f4cd00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:53.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/47926242 shutdown_connections 2023-07-14T19:26:53.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.562+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/47926242 wait complete. 2023-07-14T19:26:53.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 Processor -- start 2023-07-14T19:26:53.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 -- start start 2023-07-14T19:26:53.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd0069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0100550 0x7f4cd019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd019d560 con 0x7f4cd00ffbc0 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd019d6d0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.563+0000 7f4cd4f9e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd019d890 con 0x7f4cd0100550 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0100550 0x7f4cd019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd0069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40662/0 (socket says 172.21.15.82:40662) 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cceffd700 1 -- 172.21.15.82:0/2039105782 learned_addr learned my addr 172.21.15.82:0/2039105782 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 -- 172.21.15.82:0/2039105782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0100550 msgr2=0x7f4cd019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0100550 0x7f4cd019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 -- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 msgr2=0x7f4cd019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 -- 172.21.15.82:0/2039105782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00095f0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cceffd700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4ccf7fe700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd0069490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b620 tx=0x7f4cc0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc001d070 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd019db10 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc0022470 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd019df90 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.564+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0017650 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.565+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd0105fc0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.567+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4cc00225e0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.567+0000 7f4cccff9700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4cb80a11f0 0x7f4cb80a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.567+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4cc00cdf20 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.567+0000 7f4cceffd700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4cb80a11f0 0x7f4cb80a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.568+0000 7f4cceffd700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4cb80a11f0 0x7f4cb80a36a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0069200 tx=0x7f4cc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:53.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.572+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4cc0097ba0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:53 smithi082 ceph-mon[127461]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:53.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:53 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3516117034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:53.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:53 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1529311032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:53.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.769+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cd002c8e0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.770+0000 7f4cccff9700 1 -- 172.21.15.82:0/2039105782 <== mon.2 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cc009b1c0 con 0x7f4cd00fe9d0 2023-07-14T19:26:53.771 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:26:53.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4cb80a11f0 msgr2=0x7f4cb80a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4cb80a11f0 0x7f4cb80a36a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0069200 tx=0x7f4cc4009260 comp rx=0 tx=0).stop 2023-07-14T19:26:53.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd00fe9d0 msgr2=0x7f4cd0069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd0069490 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b620 tx=0x7f4cc0004730 comp rx=0 tx=0).stop 2023-07-14T19:26:53.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 shutdown_connections 2023-07-14T19:26:53.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0100550 0x7f4cd019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd00fe9d0 0x7f4cd0069490 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4cb80a11f0 0x7f4cb80a36a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 --2- 172.21.15.82:0/2039105782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cd00ffbc0 0x7f4cd019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 >> 172.21.15.82:0/2039105782 conn(0x7f4cd00f9fd0 msgr2=0x7f4cd0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:53.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 shutdown_connections 2023-07-14T19:26:53.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.772+0000 7f4cd4f9e700 1 -- 172.21.15.82:0/2039105782 wait complete. 2023-07-14T19:26:53.785 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-14T19:26:53.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 -- 172.21.15.82:0/3548517390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 msgr2=0x7fcde41001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 --2- 172.21.15.82:0/3548517390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde41001a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009990 tx=0x7fcdd4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 -- 172.21.15.82:0/3548517390 shutdown_connections 2023-07-14T19:26:53.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 --2- 172.21.15.82:0/3548517390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 0x7fcde4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 --2- 172.21.15.82:0/3548517390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 0x7fcde40ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 --2- 172.21.15.82:0/3548517390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde41001a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 -- 172.21.15.82:0/3548517390 >> 172.21.15.82:0/3548517390 conn(0x7fcde40f98a0 msgr2=0x7fcde40fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:53.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 -- 172.21.15.82:0/3548517390 shutdown_connections 2023-07-14T19:26:53.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.917+0000 7fcdec75f700 1 -- 172.21.15.82:0/3548517390 wait complete. 2023-07-14T19:26:53.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.918+0000 7fcdec75f700 1 Processor -- start 2023-07-14T19:26:53.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.918+0000 7fcdec75f700 1 -- start start 2023-07-14T19:26:53.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.918+0000 7fcdec75f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 0x7fcde4193760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde4193ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 0x7fcde419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde419b2e0 con 0x7fcde40ffd30 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde419b420 con 0x7fcde41006e0 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde419b5b0 con 0x7fcde40ff020 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde4193ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdea4fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 0x7fcde4193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde4193ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40676/0 (socket says 172.21.15.82:40676) 2023-07-14T19:26:53.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdea4fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 0x7fcde4193760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:49938/0 (socket says 172.21.15.82:49938) 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 -- 172.21.15.82:0/1114141177 learned_addr learned my addr 172.21.15.82:0/1114141177 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdeacfc700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 0x7fcde419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 -- 172.21.15.82:0/1114141177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 msgr2=0x7fcde4193760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 0x7fcde4193760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 -- 172.21.15.82:0/1114141177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 msgr2=0x7fcde419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 0x7fcde419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 -- 172.21.15.82:0/1114141177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40095f0 con 0x7fcde40ffd30 2023-07-14T19:26:53.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdeacfc700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 0x7fcde419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcde9cfa700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde4193ca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b680 tx=0x7fcdd4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd401d070 con 0x7fcde40ffd30 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde419b830 con 0x7fcde40ffd30 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdd4021410 con 0x7fcde40ffd30 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4017650 con 0x7fcde40ffd30 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.919+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde419bcb0 con 0x7fcde40ffd30 2023-07-14T19:26:53.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.920+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde41092b0 con 0x7fcde40ffd30 2023-07-14T19:26:53.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.922+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcdd4021580 con 0x7fcde40ffd30 2023-07-14T19:26:53.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.922+0000 7fcddb7fe700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcdd00a33b0 0x7fcdd00a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:53.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.922+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcdd40cdfa0 con 0x7fcde40ffd30 2023-07-14T19:26:53.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.922+0000 7fcdea4fb700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcdd00a33b0 0x7fcdd00a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:53.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.923+0000 7fcdea4fb700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcdd00a33b0 0x7fcdd00a5860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcde4067310 tx=0x7fcde0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:53.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:53.926+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fcdd4097c20 con 0x7fcde40ffd30 2023-07-14T19:26:53.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:53 smithi133 ceph-mon[128894]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:53.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:53 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3516117034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:53.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:53 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1529311032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:53.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:53 smithi161 ceph-mon[129184]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:53.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:53 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3516117034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:26:53.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:53 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1529311032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:54.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.129+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcde419c2f0 con 0x7fcde40ffd30 2023-07-14T19:26:54.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.129+0000 7fcddb7fe700 1 -- 172.21.15.82:0/1114141177 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcdd409b240 con 0x7fcde40ffd30 2023-07-14T19:26:54.130 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:54.130 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2023-07-14T19:26:54.130 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:26:54.130 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2023-07-14T19:26:54.131 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2023-07-14T19:26:54.131 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:26:54.131 INFO:teuthology.orchestra.run.smithi082.stdout:] 2023-07-14T19:26:54.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcdd00a33b0 msgr2=0x7fcdd00a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:54.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcdd00a33b0 0x7fcdd00a5860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcde4067310 tx=0x7fcde0007400 comp rx=0 tx=0).stop 2023-07-14T19:26:54.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 msgr2=0x7fcde4193ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:54.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde4193ca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b680 tx=0x7fcdd4005740 comp rx=0 tx=0).stop 2023-07-14T19:26:54.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 shutdown_connections 2023-07-14T19:26:54.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde41006e0 0x7fcde419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcde40ff020 0x7fcde4193760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcdd00a33b0 0x7fcdd00a5860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 --2- 172.21.15.82:0/1114141177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcde40ffd30 0x7fcde4193ca0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 >> 172.21.15.82:0/1114141177 conn(0x7fcde40f98a0 msgr2=0x7fcde40fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:54.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 shutdown_connections 2023-07-14T19:26:54.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.131+0000 7fcdec75f700 1 -- 172.21.15.82:0/1114141177 wait complete. 2023-07-14T19:26:54.145 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-14T19:26:54.145 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-14T19:26:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 -- 172.21.15.82:0/1338279012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c102fe0 msgr2=0x7f4f1c1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:54.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 --2- 172.21.15.82:0/1338279012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c102fe0 0x7f4f1c1053c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009990 tx=0x7f4f0c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 -- 172.21.15.82:0/1338279012 shutdown_connections 2023-07-14T19:26:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 --2- 172.21.15.82:0/1338279012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1c105900 0x7f4f1c107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 --2- 172.21.15.82:0/1338279012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c1006c0 0x7f4f1c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 --2- 172.21.15.82:0/1338279012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c102fe0 0x7f4f1c1053c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.272+0000 7f4f247b8700 1 -- 172.21.15.82:0/1338279012 >> 172.21.15.82:0/1338279012 conn(0x7f4f1c0fa160 msgr2=0x7f4f1c0fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:54.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 -- 172.21.15.82:0/1338279012 shutdown_connections 2023-07-14T19:26:54.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 -- 172.21.15.82:0/1338279012 wait complete. 2023-07-14T19:26:54.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 Processor -- start 2023-07-14T19:26:54.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 -- start start 2023-07-14T19:26:54.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c1006c0 0x7f4f1c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:54.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1c102fe0 0x7f4f1c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:54.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 0x7f4f1c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:54.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c19d480 con 0x7f4f1c105900 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c19d5c0 con 0x7f4f1c1006c0 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f247b8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c19d750 con 0x7f4f1c102fe0 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f22d55700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 0x7f4f1c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f22d55700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 0x7f4f1c19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40692/0 (socket says 172.21.15.82:40692) 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f22d55700 1 -- 172.21.15.82:0/3158167253 learned_addr learned my addr 172.21.15.82:0/3158167253 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f22554700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c1006c0 0x7f4f1c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:54.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.273+0000 7f4f21d53700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1c102fe0 0x7f4f1c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f22d55700 1 -- 172.21.15.82:0/3158167253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1c102fe0 msgr2=0x7f4f1c195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f22d55700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1c102fe0 0x7f4f1c195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f22d55700 1 -- 172.21.15.82:0/3158167253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c1006c0 msgr2=0x7f4f1c195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f22d55700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c1006c0 0x7f4f1c195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f22d55700 1 -- 172.21.15.82:0/3158167253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0095f0 con 0x7f4f1c105900 2023-07-14T19:26:54.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f22d55700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 0x7f4f1c19ceb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007ee0 tx=0x7f4f1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14017070 con 0x7f4f1c105900 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c0fe510 con 0x7f4f1c105900 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1400f040 con 0x7f4f1c105900 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14005580 con 0x7f4f1c105900 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.274+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c0fe9d0 con 0x7f4f1c105900 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.275+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c0fbcd0 con 0x7f4f1c105900 2023-07-14T19:26:54.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.276+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f1400db20 con 0x7f4f1c105900 2023-07-14T19:26:54.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.277+0000 7f4f137fe700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f080a1150 0x7f4f080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:54.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.277+0000 7f4f22554700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f080a1150 0x7f4f080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:54.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.277+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f140cbb80 con 0x7f4f1c105900 2023-07-14T19:26:54.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.277+0000 7f4f22554700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f080a1150 0x7f4f080a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f18007c60 tx=0x7f4f180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:54.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.281+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4f14095770 con 0x7f4f1c105900 2023-07-14T19:26:54.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:54.482+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4f1c02c8e0 con 0x7f4f1c105900 2023-07-14T19:26:54.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2039105782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:54.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1114141177' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T19:26:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:54.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:54 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2039105782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:55.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1114141177' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T19:26:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:55.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:54 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2039105782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:26:55.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1114141177' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-14T19:26:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:55.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:55.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:54 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.483+0000 7f4f137fe700 1 -- 172.21.15.82:0/3158167253 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4f14098d90 con 0x7f4f1c105900 2023-07-14T19:26:55.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f080a1150 msgr2=0x7f4f080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f080a1150 0x7f4f080a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4f18007c60 tx=0x7f4f180073d0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 msgr2=0x7f4f1c19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 0x7f4f1c19ceb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007ee0 tx=0x7f4f1400d3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 shutdown_connections 2023-07-14T19:26:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f080a1150 0x7f4f080a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f1c105900 0x7f4f1c19ceb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c1006c0 0x7f4f1c195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 --2- 172.21.15.82:0/3158167253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f1c102fe0 0x7f4f1c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 >> 172.21.15.82:0/3158167253 conn(0x7f4f1c0fa160 msgr2=0x7f4f1c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:55.489 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 shutdown_connections 2023-07-14T19:26:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.487+0000 7f4f247b8700 1 -- 172.21.15.82:0/3158167253 wait complete. 2023-07-14T19:26:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2023-07-14T19:26:55.501 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-14T19:26:55.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.631+0000 7f5361a25700 1 -- 172.21.15.82:0/3617489828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 msgr2=0x7f535c0ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.631+0000 7f5361a25700 1 --2- 172.21.15.82:0/3617489828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c0ff2a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5350009990 tx=0x7f5350009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 -- 172.21.15.82:0/3617489828 shutdown_connections 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 --2- 172.21.15.82:0/3617489828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0ff7e0 0x7f535c0ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 --2- 172.21.15.82:0/3617489828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535c0fe4e0 0x7f535c0fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 --2- 172.21.15.82:0/3617489828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c0ff2a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 -- 172.21.15.82:0/3617489828 >> 172.21.15.82:0/3617489828 conn(0x7f535c0fa130 msgr2=0x7f535c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 -- 172.21.15.82:0/3617489828 shutdown_connections 2023-07-14T19:26:55.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 -- 172.21.15.82:0/3617489828 wait complete. 2023-07-14T19:26:55.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.632+0000 7f5361a25700 1 Processor -- start 2023-07-14T19:26:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 -- start start 2023-07-14T19:26:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 0x7f535c197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535c0ff7e0 0x7f535c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c19f850 con 0x7f535c0fee30 2023-07-14T19:26:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c06a090 con 0x7f535c0ff7e0 2023-07-14T19:26:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f5361a25700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c06a250 con 0x7f535c0fe4e0 2023-07-14T19:26:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f535affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 0x7f535c197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f535affd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 0x7f535c197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:49966/0 (socket says 172.21.15.82:49966) 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f535affd700 1 -- 172.21.15.82:0/1576162980 learned_addr learned my addr 172.21.15.82:0/1576162980 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f535b7fe700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535c0ff7e0 0x7f535c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f535affd700 1 -- 172.21.15.82:0/1576162980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535c0ff7e0 msgr2=0x7f535c19f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.633+0000 7f535affd700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535c0ff7e0 0x7f535c19f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f535affd700 1 -- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 msgr2=0x7f535c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f535a7fc700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f535affd700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f535affd700 1 -- 172.21.15.82:0/1576162980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53500095f0 con 0x7f535c0fe4e0 2023-07-14T19:26:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f535a7fc700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:26:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f535affd700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 0x7f535c197d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f534c00e9a0 tx=0x7f534c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c009790 con 0x7f535c0fe4e0 2023-07-14T19:26:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f534c00cbd0 con 0x7f535c0fe4e0 2023-07-14T19:26:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c018680 con 0x7f535c0fe4e0 2023-07-14T19:26:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c06a530 con 0x7f535c0fe4e0 2023-07-14T19:26:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.634+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c06aa40 con 0x7f535c0fe4e0 2023-07-14T19:26:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.635+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c10d620 con 0x7f535c0fe4e0 2023-07-14T19:26:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.636+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f534c00cd40 con 0x7f535c0fe4e0 2023-07-14T19:26:55.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.637+0000 7f5360a23700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f53440a11d0 0x7f53440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:55.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.637+0000 7f535a7fc700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f53440a11d0 0x7f53440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:55.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.637+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f534c014070 con 0x7f535c0fe4e0 2023-07-14T19:26:55.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.637+0000 7f535a7fc700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f53440a11d0 0x7f53440a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f535000b2e0 tx=0x7f5350000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:55.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.641+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f534c096300 con 0x7f535c0fe4e0 2023-07-14T19:26:55.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:55.844+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f535c06b200 con 0x7f535c0fe4e0 2023-07-14T19:26:55.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3158167253' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T19:26:55.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:55 smithi082 ceph-mon[127461]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:55.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:55 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:55.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:55 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3158167253' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T19:26:55.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:55 smithi082 ceph-mon[127461]: osdmap e78: 12 total, 12 up, 12 in 2023-07-14T19:26:55.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:55 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3158167253' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T19:26:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:55 smithi161 ceph-mon[129184]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:55 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:55 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:55 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3158167253' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T19:26:55.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:55 smithi161 ceph-mon[129184]: osdmap e78: 12 total, 12 up, 12 in 2023-07-14T19:26:56.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3158167253' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-14T19:26:56.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:55 smithi133 ceph-mon[128894]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:56.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:26:56.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:55 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:56.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3158167253' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-14T19:26:56.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:55 smithi133 ceph-mon[128894]: osdmap e78: 12 total, 12 up, 12 in 2023-07-14T19:26:56.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.491+0000 7f5360a23700 1 -- 172.21.15.82:0/1576162980 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f534c099920 con 0x7f535c0fe4e0 2023-07-14T19:26:56.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.504+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f53440a11d0 msgr2=0x7f53440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:56.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.504+0000 7f5361a25700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f53440a11d0 0x7f53440a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f535000b2e0 tx=0x7f5350000bc0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.504+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 msgr2=0x7f535c197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.504+0000 7f5361a25700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 0x7f535c197d10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f534c00e9a0 tx=0x7f534c00ecb0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 shutdown_connections 2023-07-14T19:26:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535c0ff7e0 0x7f535c19f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f535c0fe4e0 0x7f535c197d10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f53440a11d0 0x7f53440a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 --2- 172.21.15.82:0/1576162980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535c0fee30 0x7f535c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 >> 172.21.15.82:0/1576162980 conn(0x7f535c0fa130 msgr2=0x7f535c0fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 shutdown_connections 2023-07-14T19:26:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.505+0000 7f5361a25700 1 -- 172.21.15.82:0/1576162980 wait complete. 2023-07-14T19:26:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2023-07-14T19:26:56.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-14T19:26:56.650 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 -- 172.21.15.82:0/4003642209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 msgr2=0x7ff9600fe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 --2- 172.21.15.82:0/4003642209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9600fe6d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff950009990 tx=0x7ff950009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 -- 172.21.15.82:0/4003642209 shutdown_connections 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 --2- 172.21.15.82:0/4003642209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9600ffe40 0x7ff960106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 --2- 172.21.15.82:0/4003642209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9600fe6d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 --2- 172.21.15.82:0/4003642209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9600ff4b0 0x7ff9600ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 -- 172.21.15.82:0/4003642209 >> 172.21.15.82:0/4003642209 conn(0x7ff9600f98e0 msgr2=0x7ff9600fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:56.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 -- 172.21.15.82:0/4003642209 shutdown_connections 2023-07-14T19:26:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.648+0000 7ff9663b7700 1 -- 172.21.15.82:0/4003642209 wait complete. 2023-07-14T19:26:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 Processor -- start 2023-07-14T19:26:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 -- start start 2023-07-14T19:26:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9601958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9600ff4b0 0x7ff960195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:56.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9600ffe40 0x7ff96019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96019d440 con 0x7ff9600fe2c0 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96019d580 con 0x7ff9600ffe40 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff9663b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96019d710 con 0x7ff9600ff4b0 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9601958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9601958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40718/0 (socket says 172.21.15.82:40718) 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff95ffff700 1 -- 172.21.15.82:0/515904993 learned_addr learned my addr 172.21.15.82:0/515904993 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:56.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff957fff700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9600ff4b0 0x7ff960195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.649+0000 7ff964954700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9600ffe40 0x7ff96019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95ffff700 1 -- 172.21.15.82:0/515904993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9600ff4b0 msgr2=0x7ff960195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95ffff700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9600ff4b0 0x7ff960195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95ffff700 1 -- 172.21.15.82:0/515904993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9600ffe40 msgr2=0x7ff96019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95ffff700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9600ffe40 0x7ff96019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95ffff700 1 -- 172.21.15.82:0/515904993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500095f0 con 0x7ff9600fe2c0 2023-07-14T19:26:56.654 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95ffff700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9601958c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff9500059d0 tx=0x7ff950005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:56.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95001d070 con 0x7ff9600fe2c0 2023-07-14T19:26:56.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff950004020 con 0x7ff9600fe2c0 2023-07-14T19:26:56.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96019d990 con 0x7ff9600fe2c0 2023-07-14T19:26:56.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950022c40 con 0x7ff9600fe2c0 2023-07-14T19:26:56.655 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.650+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96019ddb0 con 0x7ff9600fe2c0 2023-07-14T19:26:56.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.651+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff960109260 con 0x7ff9600fe2c0 2023-07-14T19:26:56.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.652+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff950022470 con 0x7ff9600fe2c0 2023-07-14T19:26:56.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.652+0000 7ff95dffb700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff94c0a11d0 0x7ff94c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:56.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.653+0000 7ff957fff700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff94c0a11d0 0x7ff94c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:56.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.653+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7ff9500cdd10 con 0x7ff9600fe2c0 2023-07-14T19:26:56.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.653+0000 7ff957fff700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff94c0a11d0 0x7ff94c0a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff94800ac50 tx=0x7ff94800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:56.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.657+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff950097600 con 0x7ff9600fe2c0 2023-07-14T19:26:56.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:56.907+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff960109420 con 0x7ff9600fe2c0 2023-07-14T19:26:56.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:56 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1576162980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T19:26:56.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:56 smithi082 ceph-mon[127461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T19:26:56.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:56 smithi082 ceph-mon[127461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T19:26:56.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:56 smithi082 ceph-mon[127461]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T19:26:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:56 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1576162980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T19:26:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:56 smithi161 ceph-mon[129184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T19:26:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:56 smithi161 ceph-mon[129184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T19:26:56.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:56 smithi161 ceph-mon[129184]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T19:26:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:56 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1576162980' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T19:26:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:56 smithi133 ceph-mon[128894]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-14T19:26:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:56 smithi133 ceph-mon[128894]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-14T19:26:57.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:56 smithi133 ceph-mon[128894]: osdmap e79: 12 total, 12 up, 12 in 2023-07-14T19:26:57.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.740+0000 7ff95dffb700 1 -- 172.21.15.82:0/515904993 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff960109420 con 0x7ff9600fe2c0 2023-07-14T19:26:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.742+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff94c0a11d0 msgr2=0x7ff94c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.742+0000 7ff9663b7700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff94c0a11d0 0x7ff94c0a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff94800ac50 tx=0x7ff94800a380 comp rx=0 tx=0).stop 2023-07-14T19:26:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.742+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 msgr2=0x7ff9601958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.742+0000 7ff9663b7700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9601958c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff9500059d0 tx=0x7ff950005740 comp rx=0 tx=0).stop 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 shutdown_connections 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9600ffe40 0x7ff96019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff94c0a11d0 0x7ff94c0a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9600fe2c0 0x7ff9601958c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 --2- 172.21.15.82:0/515904993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9600ff4b0 0x7ff960195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 >> 172.21.15.82:0/515904993 conn(0x7ff9600f98e0 msgr2=0x7ff960104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 shutdown_connections 2023-07-14T19:26:57.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.743+0000 7ff9663b7700 1 -- 172.21.15.82:0/515904993 wait complete. 2023-07-14T19:26:57.748 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2023-07-14T19:26:57.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:26:57.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 -- 172.21.15.82:0/915088695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3620100e20 msgr2=0x7f3620101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 --2- 172.21.15.82:0/915088695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3620100e20 0x7f3620101270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3610009990 tx=0x7f3610009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 -- 172.21.15.82:0/915088695 shutdown_connections 2023-07-14T19:26:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 --2- 172.21.15.82:0/915088695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36201017b0 0x7f3620108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 --2- 172.21.15.82:0/915088695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36200ffc30 0x7f3620100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 --2- 172.21.15.82:0/915088695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3620100e20 0x7f3620101270 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.880 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 -- 172.21.15.82:0/915088695 >> 172.21.15.82:0/915088695 conn(0x7f36200fb210 msgr2=0x7f36200fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 -- 172.21.15.82:0/915088695 shutdown_connections 2023-07-14T19:26:57.881 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 -- 172.21.15.82:0/915088695 wait complete. 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.879+0000 7f3624c29700 1 Processor -- start 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 -- start start 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36200ffc30 0x7f3620195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3620100e20 0x7f3620195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 0x7f362019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362019d4f0 con 0x7f36201017b0 2023-07-14T19:26:57.883 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362019d630 con 0x7f3620100e20 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362019d7c0 con 0x7f36200ffc30 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 0x7f362019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 0x7f362019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40740/0 (socket says 172.21.15.82:40740) 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 -- 172.21.15.82:0/3674982010 learned_addr learned my addr 172.21.15.82:0/3674982010 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361e59c700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36200ffc30 0x7f3620195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361dd9b700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3620100e20 0x7f3620195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:57.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 -- 172.21.15.82:0/3674982010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36200ffc30 msgr2=0x7f3620195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36200ffc30 0x7f3620195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 -- 172.21.15.82:0/3674982010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3620100e20 msgr2=0x7f3620195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3620100e20 0x7f3620195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 -- 172.21.15.82:0/3674982010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100095f0 con 0x7f36201017b0 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361dd9b700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3620100e20 0x7f3620195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f361ed9d700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 0x7f362019cf20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3618007ee0 tx=0x7f361800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800ddd0 con 0x7f36201017b0 2023-07-14T19:26:57.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f361800f040 con 0x7f36201017b0 2023-07-14T19:26:57.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362019daa0 con 0x7f36201017b0 2023-07-14T19:26:57.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618015400 con 0x7f36201017b0 2023-07-14T19:26:57.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.880+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362019df60 con 0x7f36201017b0 2023-07-14T19:26:57.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.881+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3620107120 con 0x7f36201017b0 2023-07-14T19:26:57.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.882+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3618004000 con 0x7f36201017b0 2023-07-14T19:26:57.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.883+0000 7f36177fe700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f360c0a11a0 0x7f360c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:57.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.883+0000 7f361e59c700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f360c0a11a0 0x7f360c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:57.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.883+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36180cc8e0 con 0x7f36201017b0 2023-07-14T19:26:57.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.883+0000 7f361e59c700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f360c0a11a0 0x7f360c0a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3620100c80 tx=0x7f3608009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:57.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:57.886+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3618096180 con 0x7f36201017b0 2023-07-14T19:26:57.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: pgmap v245: 97 pgs: 11 creating+peering, 6 active+clean, 80 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:57.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/515904993' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T19:26:57.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T19:26:57.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T19:26:57.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:57.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:57.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.078+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f362002cb30 con 0x7f36201017b0 2023-07-14T19:26:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.079+0000 7f36177fe700 1 -- 172.21.15.82:0/3674982010 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+22142 (secure 0 0 0) 0x7f3618023090 con 0x7f36201017b0 2023-07-14T19:26:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:58.081 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:26:56.908206+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:26:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f360c0a11a0 msgr2=0x7f360c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f360c0a11a0 0x7f360c0a3650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3620100c80 tx=0x7f3608009260 comp rx=0 tx=0).stop 2023-07-14T19:26:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 msgr2=0x7f362019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 0x7f362019cf20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3618007ee0 tx=0x7f361800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 shutdown_connections 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f360c0a11a0 0x7f360c0a3650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36201017b0 0x7f362019cf20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36200ffc30 0x7f3620195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 --2- 172.21.15.82:0/3674982010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3620100e20 0x7f3620195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 >> 172.21.15.82:0/3674982010 conn(0x7f36200fb210 msgr2=0x7f3620104260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.081+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 shutdown_connections 2023-07-14T19:26:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.082+0000 7f3624c29700 1 -- 172.21.15.82:0/3674982010 wait complete. 2023-07-14T19:26:58.093 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: pgmap v245: 97 pgs: 11 creating+peering, 6 active+clean, 80 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:58.093 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/515904993' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T19:26:58.093 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T19:26:58.093 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:58.094 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:58.095 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:26:58.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: pgmap v245: 97 pgs: 11 creating+peering, 6 active+clean, 80 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:26:58.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/515904993' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-14T19:26:58.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-14T19:26:58.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: osdmap e80: 12 total, 12 up, 12 in 2023-07-14T19:26:58.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:26:58.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:26:58.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.218+0000 7fe750857700 1 -- 172.21.15.82:0/1128651606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 msgr2=0x7fe74c1081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.218+0000 7fe750857700 1 --2- 172.21.15.82:0/1128651606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c1081b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b0a0 tx=0x7fe74400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 -- 172.21.15.82:0/1128651606 shutdown_connections 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 --2- 172.21.15.82:0/1128651606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c1081b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 --2- 172.21.15.82:0/1128651606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c0fdef0 0x7fe74c0fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 --2- 172.21.15.82:0/1128651606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74c100f60 0x7fe74c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 -- 172.21.15.82:0/1128651606 >> 172.21.15.82:0/1128651606 conn(0x7fe74c0f98b0 msgr2=0x7fe74c0fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 -- 172.21.15.82:0/1128651606 shutdown_connections 2023-07-14T19:26:58.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 -- 172.21.15.82:0/1128651606 wait complete. 2023-07-14T19:26:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 Processor -- start 2023-07-14T19:26:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.219+0000 7fe750857700 1 -- start start 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74c0fdef0 0x7fe74c1959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c100f60 0x7fe74c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c19d570 con 0x7fe74c1018f0 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c19d6b0 con 0x7fe74c100f60 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c19d840 con 0x7fe74c0fdef0 2023-07-14T19:26:58.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40754/0 (socket says 172.21.15.82:40754) 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 -- 172.21.15.82:0/3954630822 learned_addr learned my addr 172.21.15.82:0/3954630822 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe749d9b700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c100f60 0x7fe74c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74a59c700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74c0fdef0 0x7fe74c1959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 -- 172.21.15.82:0/3954630822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74c0fdef0 msgr2=0x7fe74c1959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74c0fdef0 0x7fe74c1959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 -- 172.21.15.82:0/3954630822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c100f60 msgr2=0x7fe74c195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c100f60 0x7fe74c195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 -- 172.21.15.82:0/3954630822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744009d00 con 0x7fe74c1018f0 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe74ad9d700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c19cfa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b780 tx=0x7fe74400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400e040 con 0x7fe74c1018f0 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe744004e60 con 0x7fe74c1018f0 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c19dac0 con 0x7fe74c1018f0 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74401b740 con 0x7fe74c1018f0 2023-07-14T19:26:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.220+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c19df40 con 0x7fe74c1018f0 2023-07-14T19:26:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.221+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c10b410 con 0x7fe74c1018f0 2023-07-14T19:26:58.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.222+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe74401da40 con 0x7fe74c1018f0 2023-07-14T19:26:58.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.223+0000 7fe73b7fe700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7340a11a0 0x7fe7340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.223+0000 7fe74a59c700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7340a11a0 0x7fe7340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.223+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7440ce980 con 0x7fe74c1018f0 2023-07-14T19:26:58.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.224+0000 7fe74a59c700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7340a11a0 0x7fe7340a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe74c068720 tx=0x7fe73c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:58.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.227+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe7440982b0 con 0x7fe74c1018f0 2023-07-14T19:26:58.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.416+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe74c106b60 con 0x7fe74c1018f0 2023-07-14T19:26:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.417+0000 7fe73b7fe700 1 -- 172.21.15.82:0/3954630822 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22129 (secure 0 0 0) 0x7fe74409b8d0 con 0x7fe74c1018f0 2023-07-14T19:26:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:26:58.418 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:26:58.303918+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7340a11a0 msgr2=0x7fe7340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7340a11a0 0x7fe7340a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe74c068720 tx=0x7fe73c00b5b0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 msgr2=0x7fe74c19cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c19cfa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b780 tx=0x7fe74400bd00 comp rx=0 tx=0).stop 2023-07-14T19:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 shutdown_connections 2023-07-14T19:26:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7340a11a0 0x7fe7340a3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe74c1018f0 0x7fe74c19cfa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe74c0fdef0 0x7fe74c1959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 --2- 172.21.15.82:0/3954630822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe74c100f60 0x7fe74c195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 >> 172.21.15.82:0/3954630822 conn(0x7fe74c0f98b0 msgr2=0x7fe74c0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 shutdown_connections 2023-07-14T19:26:58.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.419+0000 7fe750857700 1 -- 172.21.15.82:0/3954630822 wait complete. 2023-07-14T19:26:58.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-14T19:26:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 -- 172.21.15.82:0/1986923946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c100690 msgr2=0x7f5b0c106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1986923946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c100690 0x7f5b0c106f50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400b0a0 tx=0x7f5b0400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 -- 172.21.15.82:0/1986923946 shutdown_connections 2023-07-14T19:26:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1986923946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c100690 0x7f5b0c106f50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1986923946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 0x7f5b0c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1986923946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 -- 172.21.15.82:0/1986923946 >> 172.21.15.82:0/1986923946 conn(0x7f5b0c0fa130 msgr2=0x7f5b0c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 -- 172.21.15.82:0/1986923946 shutdown_connections 2023-07-14T19:26:58.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 -- 172.21.15.82:0/1986923946 wait complete. 2023-07-14T19:26:58.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.556+0000 7f5b11c97700 1 Processor -- start 2023-07-14T19:26:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 -- start start 2023-07-14T19:26:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 0x7f5b0c068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c100690 0x7f5b0c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19d5f0 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19d760 con 0x7f5b0c100690 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b11c97700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c19d920 con 0x7f5b0c0feb10 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40766/0 (socket says 172.21.15.82:40766) 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 -- 172.21.15.82:0/1677771240 learned_addr learned my addr 172.21.15.82:0/1677771240 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 0x7f5b0c068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0bfff700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c100690 0x7f5b0c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 -- 172.21.15.82:0/1677771240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 msgr2=0x7f5b0c068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 0x7f5b0c068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 -- 172.21.15.82:0/1677771240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c100690 msgr2=0x7f5b0c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c100690 0x7f5b0c19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 -- 172.21.15.82:0/1677771240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04009d00 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0bfff700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c100690 0x7f5b0c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:26:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 0x7f5b0c068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:26:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.557+0000 7f5b0affd700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c19a810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000d980 tx=0x7f5b0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.558+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00004220 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.558+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b00007640 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.558+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c19dc00 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.558+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005610 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.558+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c19e110 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.559+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c106050 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.560+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b00004750 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.560+0000 7f5b08ff9700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5af40a1150 0x7f5af40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.560+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5af40a1150 0x7f5af40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.560+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b000ccb80 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.561+0000 7f5b0b7fe700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5af40a1150 0x7f5af40a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0ffb60 tx=0x7f5afc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.564+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5b00096380 con 0x7f5b0c0ffd00 2023-07-14T19:26:58.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:58.800+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5b0c02c8e0 con 0x7f5b0c0ffd00 2023-07-14T19:26:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:58 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/515904993' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T19:26:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:58 smithi133 ceph-mon[128894]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T19:26:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:58 smithi133 ceph-mon[128894]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T19:26:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:58 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3674982010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:58 smithi133 ceph-mon[128894]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T19:26:59.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:58 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3954630822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:58 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/515904993' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T19:26:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:58 smithi082 ceph-mon[127461]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T19:26:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:58 smithi082 ceph-mon[127461]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T19:26:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:58 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3674982010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:58 smithi082 ceph-mon[127461]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T19:26:59.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:58 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3954630822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:59.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:58 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/515904993' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-14T19:26:59.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:58 smithi161 ceph-mon[129184]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-14T19:26:59.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:58 smithi161 ceph-mon[129184]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-14T19:26:59.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:58 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3674982010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:59.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:58 smithi161 ceph-mon[129184]: osdmap e81: 12 total, 12 up, 12 in 2023-07-14T19:26:59.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:58 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3954630822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:26:59.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.753+0000 7f5b08ff9700 1 -- 172.21.15.82:0/1677771240 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5b000160b0 con 0x7f5b0c0ffd00 2023-07-14T19:26:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5af40a1150 msgr2=0x7f5af40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5af40a1150 0x7f5af40a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c0ffb60 tx=0x7f5afc009260 comp rx=0 tx=0).stop 2023-07-14T19:26:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 msgr2=0x7f5b0c19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c19a810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000d980 tx=0x7f5b0000dc90 comp rx=0 tx=0).stop 2023-07-14T19:26:59.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 shutdown_connections 2023-07-14T19:26:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b0c100690 0x7f5b0c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0c0feb10 0x7f5b0c068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5af40a1150 0x7f5af40a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 --2- 172.21.15.82:0/1677771240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b0c0ffd00 0x7f5b0c19a810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 >> 172.21.15.82:0/1677771240 conn(0x7f5b0c0fa130 msgr2=0x7f5b0c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:59.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 shutdown_connections 2023-07-14T19:26:59.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.756+0000 7f5b11c97700 1 -- 172.21.15.82:0/1677771240 wait complete. 2023-07-14T19:26:59.772 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-07-14T19:26:59.772 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-14T19:26:59.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 -- 172.21.15.82:0/227951145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 msgr2=0x7f6990100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:59.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 --2- 172.21.15.82:0/227951145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990100150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6980009990 tx=0x7f6980009ca0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 -- 172.21.15.82:0/227951145 shutdown_connections 2023-07-14T19:26:59.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 --2- 172.21.15.82:0/227951145 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6990100690 0x7f6990106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 --2- 172.21.15.82:0/227951145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69900feb10 0x7f69900fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 --2- 172.21.15.82:0/227951145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990100150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.902+0000 7f699575b700 1 -- 172.21.15.82:0/227951145 >> 172.21.15.82:0/227951145 conn(0x7f69900fa130 msgr2=0x7f69900fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:26:59.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 -- 172.21.15.82:0/227951145 shutdown_connections 2023-07-14T19:26:59.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 -- 172.21.15.82:0/227951145 wait complete. 2023-07-14T19:26:59.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 Processor -- start 2023-07-14T19:26:59.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 -- start start 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69900feb10 0x7f69901949c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6990100690 0x7f699019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019c540 con 0x7f69900ffd00 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019c680 con 0x7f69900feb10 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.903+0000 7f699575b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019c810 con 0x7f6990100690 2023-07-14T19:26:59.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990194f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40780/0 (socket says 172.21.15.82:40780) 2023-07-14T19:26:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 -- 172.21.15.82:0/3402527056 learned_addr learned my addr 172.21.15.82:0/3402527056 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:26:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698f7fe700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6990100690 0x7f699019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698effd700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69900feb10 0x7f69901949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 -- 172.21.15.82:0/3402527056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6990100690 msgr2=0x7f699019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:59.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6990100690 0x7f699019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 -- 172.21.15.82:0/3402527056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69900feb10 msgr2=0x7f69901949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:26:59.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69900feb10 0x7f69901949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 -- 172.21.15.82:0/3402527056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69800095f0 con 0x7f69900ffd00 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f698e7fc700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990194f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6980004760 tx=0x7f6980004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698001d070 con 0x7f69900ffd00 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699019ca90 con 0x7f69900ffd00 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69800041d0 con 0x7f69900ffd00 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69800053a0 con 0x7f69900ffd00 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.904+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699019cfb0 con 0x7f69900ffd00 2023-07-14T19:26:59.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.905+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6990105ac0 con 0x7f69900ffd00 2023-07-14T19:26:59.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.907+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6980005500 con 0x7f69900ffd00 2023-07-14T19:26:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.907+0000 7f6977fff700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f69780a1150 0x7f69780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:26:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.907+0000 7f698effd700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f69780a1150 0x7f69780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:26:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.907+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69800cdef0 con 0x7f69900ffd00 2023-07-14T19:26:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.908+0000 7f698effd700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f69780a1150 0x7f69780a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6984006fd0 tx=0x7f6984008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:26:59.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:26:59.911+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6980097820 con 0x7f69900ffd00 2023-07-14T19:27:00.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.142+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f699002cb30 con 0x7f69900ffd00 2023-07-14T19:27:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:59 smithi133 ceph-mon[128894]: pgmap v248: 97 pgs: 34 creating+peering, 42 active+clean, 21 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2023-07-14T19:27:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:59 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] up:active 2023-07-14T19:27:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:59 smithi133 ceph-mon[128894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:59 smithi133 ceph-mon[128894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:26:59 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1677771240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T19:27:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:59 smithi082 ceph-mon[127461]: pgmap v248: 97 pgs: 34 creating+peering, 42 active+clean, 21 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2023-07-14T19:27:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:59 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] up:active 2023-07-14T19:27:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:59 smithi082 ceph-mon[127461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:00.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:59 smithi082 ceph-mon[127461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:00.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:26:59 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1677771240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T19:27:00.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:59 smithi161 ceph-mon[129184]: pgmap v248: 97 pgs: 34 creating+peering, 42 active+clean, 21 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2023-07-14T19:27:00.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:59 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] up:active 2023-07-14T19:27:00.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:59 smithi161 ceph-mon[129184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:00.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:59 smithi161 ceph-mon[129184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:00.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:26:59 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1677771240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-14T19:27:00.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.761+0000 7f6977fff700 1 -- 172.21.15.82:0/3402527056 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f698009ae40 con 0x7f69900ffd00 2023-07-14T19:27:00.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f69780a1150 msgr2=0x7f69780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f69780a1150 0x7f69780a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6984006fd0 tx=0x7f6984008040 comp rx=0 tx=0).stop 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 msgr2=0x7f6990194f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990194f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6980004760 tx=0x7f6980004790 comp rx=0 tx=0).stop 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 shutdown_connections 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6990100690 0x7f699019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69900feb10 0x7f69901949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f69780a1150 0x7f69780a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 --2- 172.21.15.82:0/3402527056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69900ffd00 0x7f6990194f00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.763+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 >> 172.21.15.82:0/3402527056 conn(0x7f69900fa130 msgr2=0x7f69900fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:00.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.764+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 shutdown_connections 2023-07-14T19:27:00.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.764+0000 7f699575b700 1 -- 172.21.15.82:0/3402527056 wait complete. 2023-07-14T19:27:00.780 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-14T19:27:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 -- 172.21.15.82:0/3814559020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48100e00 msgr2=0x7fce48101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:00.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 --2- 172.21.15.82:0/3814559020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48100e00 0x7fce48101250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce38009990 tx=0x7fce38009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 -- 172.21.15.82:0/3814559020 shutdown_connections 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 --2- 172.21.15.82:0/3814559020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce48101790 0x7fce48108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 --2- 172.21.15.82:0/3814559020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce480ffc10 0x7fce48100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 --2- 172.21.15.82:0/3814559020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48100e00 0x7fce48101250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.905+0000 7fce50361700 1 -- 172.21.15.82:0/3814559020 >> 172.21.15.82:0/3814559020 conn(0x7fce480fb250 msgr2=0x7fce480fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 -- 172.21.15.82:0/3814559020 shutdown_connections 2023-07-14T19:27:00.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 -- 172.21.15.82:0/3814559020 wait complete. 2023-07-14T19:27:00.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 Processor -- start 2023-07-14T19:27:00.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 -- start start 2023-07-14T19:27:00.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce480ffc10 0x7fce48195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:00.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce48100e00 0x7fce48195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:00.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 0x7fce4819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:00.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4819d4d0 con 0x7fce48101790 2023-07-14T19:27:00.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4819d610 con 0x7fce48100e00 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce50361700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4819d7a0 con 0x7fce480ffc10 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce4e8fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 0x7fce4819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce4e8fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 0x7fce4819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40796/0 (socket says 172.21.15.82:40796) 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce4e8fe700 1 -- 172.21.15.82:0/3517287690 learned_addr learned my addr 172.21.15.82:0/3517287690 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.906+0000 7fce4e0fd700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce480ffc10 0x7fce48195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4d8fc700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce48100e00 0x7fce48195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:00.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4e8fe700 1 -- 172.21.15.82:0/3517287690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce480ffc10 msgr2=0x7fce48195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4e8fe700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce480ffc10 0x7fce48195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4e8fe700 1 -- 172.21.15.82:0/3517287690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce48100e00 msgr2=0x7fce48195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4e8fe700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce48100e00 0x7fce48195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4e8fe700 1 -- 172.21.15.82:0/3517287690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380095f0 con 0x7fce48101790 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4d8fc700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce48100e00 0x7fce48195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce4e8fe700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 0x7fce4819cf00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce4000ef30 tx=0x7fce4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:00.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40018070 con 0x7fce48101790 2023-07-14T19:27:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4819da20 con 0x7fce48101790 2023-07-14T19:27:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce4000f040 con 0x7fce48101790 2023-07-14T19:27:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000cdb0 con 0x7fce48101790 2023-07-14T19:27:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.907+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4819df30 con 0x7fce48101790 2023-07-14T19:27:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.908+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce48106b40 con 0x7fce48101790 2023-07-14T19:27:00.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.909+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fce40003a00 con 0x7fce48101790 2023-07-14T19:27:00.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.910+0000 7fce3f7fe700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce340a1150 0x7fce340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:00.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.910+0000 7fce4e0fd700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce340a1150 0x7fce340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:00.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.910+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fce400ccd90 con 0x7fce48101790 2023-07-14T19:27:00.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.910+0000 7fce4e0fd700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce340a1150 0x7fce340a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce48100c60 tx=0x7fce4400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:00.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:00.913+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fce40096620 con 0x7fce48101790 2023-07-14T19:27:01.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.151+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fce4802ca30 con 0x7fce48101790 2023-07-14T19:27:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:00 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1677771240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T19:27:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:00 smithi133 ceph-mon[128894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:01.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:00 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3402527056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T19:27:01.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:00 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1677771240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T19:27:01.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:00 smithi082 ceph-mon[127461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:01.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:00 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3402527056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T19:27:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:00 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1677771240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-14T19:27:01.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:00 smithi161 ceph-mon[129184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:01.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:00 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3402527056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-14T19:27:01.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.768+0000 7fce3f7fe700 1 -- 172.21.15.82:0/3517287690 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fce40099c40 con 0x7fce48101790 2023-07-14T19:27:01.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce340a1150 msgr2=0x7fce340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:01.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce340a1150 0x7fce340a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce48100c60 tx=0x7fce4400b410 comp rx=0 tx=0).stop 2023-07-14T19:27:01.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 msgr2=0x7fce4819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:01.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 0x7fce4819cf00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce4000ef30 tx=0x7fce4000c3a0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 shutdown_connections 2023-07-14T19:27:01.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce340a1150 0x7fce340a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce48101790 0x7fce4819cf00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce480ffc10 0x7fce48195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 --2- 172.21.15.82:0/3517287690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce48100e00 0x7fce48195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 >> 172.21.15.82:0/3517287690 conn(0x7fce480fb250 msgr2=0x7fce480fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:01.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 shutdown_connections 2023-07-14T19:27:01.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.770+0000 7fce50361700 1 -- 172.21.15.82:0/3517287690 wait complete. 2023-07-14T19:27:01.787 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-14T19:27:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 -- 172.21.15.82:0/3346248504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a80ff6b0 msgr2=0x7f93a8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 --2- 172.21.15.82:0/3346248504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a80ff6b0 0x7f93a8106f60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b0a0 tx=0x7f93a000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 -- 172.21.15.82:0/3346248504 shutdown_connections 2023-07-14T19:27:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 --2- 172.21.15.82:0/3346248504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a80ff6b0 0x7f93a8106f60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 --2- 172.21.15.82:0/3346248504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93a81000b0 0x7f93a81004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 --2- 172.21.15.82:0/3346248504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a80fed00 0x7f93a80ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 -- 172.21.15.82:0/3346248504 >> 172.21.15.82:0/3346248504 conn(0x7f93a80f98a0 msgr2=0x7f93a80fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 -- 172.21.15.82:0/3346248504 shutdown_connections 2023-07-14T19:27:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.912+0000 7f93b050d700 1 -- 172.21.15.82:0/3346248504 wait complete. 2023-07-14T19:27:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 Processor -- start 2023-07-14T19:27:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 -- start start 2023-07-14T19:27:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a80fed00 0x7f93a8194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93a80ff6b0 0x7f93a8194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 0x7f93a819bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a819c4a0 con 0x7f93a81000b0 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a819c5e0 con 0x7f93a80ff6b0 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93b050d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a819c770 con 0x7f93a80fed00 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93aeaaa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 0x7f93a819bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93aeaaa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 0x7f93a819bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40810/0 (socket says 172.21.15.82:40810) 2023-07-14T19:27:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93aeaaa700 1 -- 172.21.15.82:0/1498749014 learned_addr learned my addr 172.21.15.82:0/1498749014 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93ae2a9700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a80fed00 0x7f93a8194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.913+0000 7f93adaa8700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93a80ff6b0 0x7f93a8194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93aeaaa700 1 -- 172.21.15.82:0/1498749014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a80fed00 msgr2=0x7f93a8194920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93aeaaa700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a80fed00 0x7f93a8194920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93aeaaa700 1 -- 172.21.15.82:0/1498749014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93a80ff6b0 msgr2=0x7f93a8194e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93aeaaa700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93a80ff6b0 0x7f93a8194e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93aeaaa700 1 -- 172.21.15.82:0/1498749014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009d00 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93aeaaa700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 0x7f93a819bed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f93a00062a0 tx=0x7f93a0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000e040 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a819c9f0 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93a000cec0 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a001baf0 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.914+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a819ce20 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.915+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a8105aa0 con 0x7f93a81000b0 2023-07-14T19:27:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.916+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f93a001bc50 con 0x7f93a81000b0 2023-07-14T19:27:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.916+0000 7f939f7fe700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93940a1150 0x7f93940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.917+0000 7f93ae2a9700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93940a1150 0x7f93940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.917+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93a00ce8a0 con 0x7f93a81000b0 2023-07-14T19:27:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.917+0000 7f93ae2a9700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93940a1150 0x7f93940a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9398005950 tx=0x7f939800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:01.920+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f93a00981d0 con 0x7f93a81000b0 2023-07-14T19:27:02.157 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.156+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f93a802c8e0 con 0x7f93a81000b0 2023-07-14T19:27:02.183 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: 2023-07-14T19:27:01.857+0000 7f0df5cde700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-14T19:27:02.183 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: 2023-07-14T19:27:01.857+0000 7f0df5cde700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-14T19:27:02.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: pgmap v249: 97 pgs: 34 creating+peering, 57 active+clean, 6 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 391 B/s wr, 4 op/s 2023-07-14T19:27:02.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T19:27:02.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3402527056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T19:27:02.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T19:27:02.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:02.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-14T19:27:02.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-14T19:27:02.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:01 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3517287690' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T19:27:02.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: pgmap v249: 97 pgs: 34 creating+peering, 57 active+clean, 6 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 391 B/s wr, 4 op/s 2023-07-14T19:27:02.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T19:27:02.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3402527056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T19:27:02.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T19:27:02.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:02.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-14T19:27:02.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-14T19:27:02.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:01 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3517287690' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T19:27:02.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: pgmap v249: 97 pgs: 34 creating+peering, 57 active+clean, 6 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 391 B/s wr, 4 op/s 2023-07-14T19:27:02.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-14T19:27:02.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3402527056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-14T19:27:02.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-14T19:27:02.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-14T19:27:02.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-14T19:27:02.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-14T19:27:02.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:01 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3517287690' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-14T19:27:02.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.523+0000 7f939f7fe700 1 -- 172.21.15.82:0/1498749014 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f93a009b7f0 con 0x7f93a81000b0 2023-07-14T19:27:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.525+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93940a1150 msgr2=0x7f93940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.525+0000 7f93b050d700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93940a1150 0x7f93940a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9398005950 tx=0x7f939800b410 comp rx=0 tx=0).stop 2023-07-14T19:27:02.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.525+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 msgr2=0x7f93a819bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.525+0000 7f93b050d700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 0x7f93a819bed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f93a00062a0 tx=0x7f93a0004d00 comp rx=0 tx=0).stop 2023-07-14T19:27:02.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 shutdown_connections 2023-07-14T19:27:02.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93940a1150 0x7f93940a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93a81000b0 0x7f93a819bed0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93a80fed00 0x7f93a8194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 --2- 172.21.15.82:0/1498749014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93a80ff6b0 0x7f93a8194e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 >> 172.21.15.82:0/1498749014 conn(0x7f93a80f98a0 msgr2=0x7f93a80fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:02.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 shutdown_connections 2023-07-14T19:27:02.530 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.526+0000 7f93b050d700 1 -- 172.21.15.82:0/1498749014 wait complete. 2023-07-14T19:27:02.540 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-07-14T19:27:02.540 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-14T19:27:02.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 -- 172.21.15.82:0/2735688973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffc20 msgr2=0x7fa22c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 --2- 172.21.15.82:0/2735688973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffc20 0x7fa22c100030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009990 tx=0x7fa21c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 -- 172.21.15.82:0/2735688973 shutdown_connections 2023-07-14T19:27:02.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 --2- 172.21.15.82:0/2735688973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c1017a0 0x7fa22c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 --2- 172.21.15.82:0/2735688973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffc20 0x7fa22c100030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 --2- 172.21.15.82:0/2735688973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c100e10 0x7fa22c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 -- 172.21.15.82:0/2735688973 >> 172.21.15.82:0/2735688973 conn(0x7fa22c0fb260 msgr2=0x7fa22c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 -- 172.21.15.82:0/2735688973 shutdown_connections 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.665+0000 7fa233d79700 1 -- 172.21.15.82:0/2735688973 wait complete. 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 Processor -- start 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 -- start start 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0ffc20 0x7fa22c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:02.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c100e10 0x7fa22c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:02.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 0x7fa22c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:02.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19d520 con 0x7fa22c1017a0 2023-07-14T19:27:02.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19d660 con 0x7fa22c100e10 2023-07-14T19:27:02.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa233d79700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19d7f0 con 0x7fa22c0ffc20 2023-07-14T19:27:02.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 0x7fa22c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:02.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 0x7fa22c19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40824/0 (socket says 172.21.15.82:40824) 2023-07-14T19:27:02.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 -- 172.21.15.82:0/2819670381 learned_addr learned my addr 172.21.15.82:0/2819670381 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:02.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa231b15700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0ffc20 0x7fa22c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:02.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa231314700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c100e10 0x7fa22c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:02.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 -- 172.21.15.82:0/2819670381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0ffc20 msgr2=0x7fa22c1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0ffc20 0x7fa22c1959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 -- 172.21.15.82:0/2819670381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c100e10 msgr2=0x7fa22c195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c100e10 0x7fa22c195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.666+0000 7fa232316700 1 -- 172.21.15.82:0/2819670381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0095f0 con 0x7fa22c1017a0 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.667+0000 7fa232316700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 0x7fa22c19cf50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b970 tx=0x7fa22400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.667+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400d860 con 0x7fa22c1017a0 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.667+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22400de80 con 0x7fa22c1017a0 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.667+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c19dad0 con 0x7fa22c1017a0 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.667+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224017400 con 0x7fa22c1017a0 2023-07-14T19:27:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.667+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c19e010 con 0x7fa22c1017a0 2023-07-14T19:27:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.668+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c10b470 con 0x7fa22c1017a0 2023-07-14T19:27:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.669+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa22400e630 con 0x7fa22c1017a0 2023-07-14T19:27:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.669+0000 7fa222ffd700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa2180a1150 0x7fa2180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.669+0000 7fa231b15700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa2180a1150 0x7fa2180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.669+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa224006e40 con 0x7fa22c1017a0 2023-07-14T19:27:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.670+0000 7fa231b15700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa2180a1150 0x7fa2180a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b5c0 tx=0x7fa21c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.673+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa224095e40 con 0x7fa22c1017a0 2023-07-14T19:27:02.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.895+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fa22c060a80 con 0x7fa2180a1150 2023-07-14T19:27:02.902 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:active 2023-07-14T19:27:02.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3517287690' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T19:27:02.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T19:27:02.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T19:27:02.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: Cluster is now healthy 2023-07-14T19:27:02.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-14T19:27:02.903 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-14T19:27:02.904 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-14T19:27:02.904 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1498749014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T19:27:02.904 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] up:active 2023-07-14T19:27:02.904 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1498749014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T19:27:02.904 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:27:02.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.936+0000 7fa222ffd700 1 -- 172.21.15.82:0/2819670381 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa22c060a80 con 0x7fa2180a1150 2023-07-14T19:27:02.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa2180a1150 msgr2=0x7fa2180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa2180a1150 0x7fa2180a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b5c0 tx=0x7fa21c005960 comp rx=0 tx=0).stop 2023-07-14T19:27:02.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 msgr2=0x7fa22c19cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:02.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 0x7fa22c19cf50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b970 tx=0x7fa22400bc80 comp rx=0 tx=0).stop 2023-07-14T19:27:02.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 shutdown_connections 2023-07-14T19:27:02.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa2180a1150 0x7fa2180a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c1017a0 0x7fa22c19cf50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa22c0ffc20 0x7fa22c1959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 --2- 172.21.15.82:0/2819670381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c100e10 0x7fa22c195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:02.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 >> 172.21.15.82:0/2819670381 conn(0x7fa22c0fb260 msgr2=0x7fa22c0fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:02.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 shutdown_connections 2023-07-14T19:27:02.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:02.938+0000 7fa233d79700 1 -- 172.21.15.82:0/2819670381 wait complete. 2023-07-14T19:27:02.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-14T19:27:03.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- 172.21.15.82:0/3996786585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160100770 msgr2=0x7fc160107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- 172.21.15.82:0/3996786585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160100770 0x7fc160107010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc15800b0a0 tx=0x7fc15800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- 172.21.15.82:0/3996786585 shutdown_connections 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- 172.21.15.82:0/3996786585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc160100770 0x7fc160107010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- 172.21.15.82:0/3996786585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1600feb30 0x7fc1600fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- 172.21.15.82:0/3996786585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 0x7fc160100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- 172.21.15.82:0/3996786585 >> 172.21.15.82:0/3996786585 conn(0x7fc1600fa150 msgr2=0x7fc1600fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- 172.21.15.82:0/3996786585 shutdown_connections 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- 172.21.15.82:0/3996786585 wait complete. 2023-07-14T19:27:03.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 Processor -- start 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- start start 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 0x7fc160195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 0x7fc160195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc160100770 0x7fc16019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16019d5c0 con 0x7fc1600feb30 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16019d700 con 0x7fc1600ffdc0 2023-07-14T19:27:03.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.075+0000 7fc168747700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16019d890 con 0x7fc160100770 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 0x7fc160195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 0x7fc160195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40850/0 (socket says 172.21.15.82:40850) 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 -- 172.21.15.82:0/604066259 learned_addr learned my addr 172.21.15.82:0/604066259 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc165ce2700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 0x7fc160195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc166ce4700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc160100770 0x7fc16019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 -- 172.21.15.82:0/604066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc160100770 msgr2=0x7fc16019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc160100770 0x7fc16019cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 -- 172.21.15.82:0/604066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 msgr2=0x7fc160195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 0x7fc160195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 -- 172.21.15.82:0/604066259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc158009d00 con 0x7fc1600feb30 2023-07-14T19:27:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc165ce2700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 0x7fc160195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1664e3700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 0x7fc160195a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc15000c920 tx=0x7fc15000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150007b00 con 0x7fc1600feb30 2023-07-14T19:27:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc150004cf0 con 0x7fc1600feb30 2023-07-14T19:27:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16019db70 con 0x7fc1600feb30 2023-07-14T19:27:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150005610 con 0x7fc1600feb30 2023-07-14T19:27:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.076+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16019e080 con 0x7fc1600feb30 2023-07-14T19:27:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.077+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16010a300 con 0x7fc1600feb30 2023-07-14T19:27:03.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.078+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc150004750 con 0x7fc1600feb30 2023-07-14T19:27:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.079+0000 7fc1577fe700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc14c0a1150 0x7fc14c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.079+0000 7fc165ce2700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc14c0a1150 0x7fc14c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.079+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc1500cc7d0 con 0x7fc1600feb30 2023-07-14T19:27:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.079+0000 7fc165ce2700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc14c0a1150 0x7fc14c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc15c005d90 tx=0x7fc15c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.082+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc150096070 con 0x7fc1600feb30 2023-07-14T19:27:03.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:active 2023-07-14T19:27:03.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3517287690' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T19:27:03.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T19:27:03.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T19:27:03.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: Cluster is now healthy 2023-07-14T19:27:03.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-14T19:27:03.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-14T19:27:03.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-14T19:27:03.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1498749014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T19:27:03.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] up:active 2023-07-14T19:27:03.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1498749014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T19:27:03.187 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:02 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:27:03.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:27:02.900+0000 7fd181693700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:27:03.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:27:02.900+0000 7fd181693700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:27:03.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:27:02.900+0000 7fd181693700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:27:03.215 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:27:02.900+0000 7fd181693700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:27:03.215 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:27:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:27:02.900+0000 7fd181693700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:27:03.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:active 2023-07-14T19:27:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3517287690' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-14T19:27:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-14T19:27:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-14T19:27:03.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: Cluster is now healthy 2023-07-14T19:27:03.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-14T19:27:03.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-14T19:27:03.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-14T19:27:03.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1498749014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-14T19:27:03.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] up:active 2023-07-14T19:27:03.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1498749014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-14T19:27:03.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:02 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:27:03.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.306+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1600690c0 con 0x7fc14c0a1150 2023-07-14T19:27:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.310+0000 7fc1577fe700 1 -- 172.21.15.82:0/604066259 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fc1600690c0 con 0x7fc14c0a1150 2023-07-14T19:27:03.311 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/_nogroup/sv_0/48aabd45-aa26-48f5-8fdd-e718fa796cac 2023-07-14T19:27:03.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc14c0a1150 msgr2=0x7fc14c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc14c0a1150 0x7fc14c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc15c005d90 tx=0x7fc15c005d00 comp rx=0 tx=0).stop 2023-07-14T19:27:03.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 msgr2=0x7fc160195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 0x7fc160195a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc15000c920 tx=0x7fc15000cce0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 shutdown_connections 2023-07-14T19:27:03.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc160100770 0x7fc16019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc14c0a1150 0x7fc14c0a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1600feb30 0x7fc160195a40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 --2- 172.21.15.82:0/604066259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1600ffdc0 0x7fc160195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 >> 172.21.15.82:0/604066259 conn(0x7fc1600fa150 msgr2=0x7fc1600689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:03.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 shutdown_connections 2023-07-14T19:27:03.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.312+0000 7fc168747700 1 -- 172.21.15.82:0/604066259 wait complete. 2023-07-14T19:27:03.325 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-14T19:27:03.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 -- 172.21.15.82:0/3847154598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100ffd70 msgr2=0x7f4e101001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 --2- 172.21.15.82:0/3847154598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100ffd70 0x7f4e101001c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009990 tx=0x7f4dfc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 -- 172.21.15.82:0/3847154598 shutdown_connections 2023-07-14T19:27:03.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 --2- 172.21.15.82:0/3847154598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e10100700 0x7f4e10106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 --2- 172.21.15.82:0/3847154598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e100feb80 0x7f4e100fef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 --2- 172.21.15.82:0/3847154598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100ffd70 0x7f4e101001c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 -- 172.21.15.82:0/3847154598 >> 172.21.15.82:0/3847154598 conn(0x7f4e100fa1a0 msgr2=0x7f4e100fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:03.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 -- 172.21.15.82:0/3847154598 shutdown_connections 2023-07-14T19:27:03.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.448+0000 7f4e17011700 1 -- 172.21.15.82:0/3847154598 wait complete. 2023-07-14T19:27:03.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 Processor -- start 2023-07-14T19:27:03.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 -- start start 2023-07-14T19:27:03.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 0x7f4e100694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e100ffd70 0x7f4e1019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e10100700 0x7f4e1019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1019d430 con 0x7f4e100feb80 2023-07-14T19:27:03.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1019d5a0 con 0x7f4e100ffd70 2023-07-14T19:27:03.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e17011700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1019d760 con 0x7f4e10100700 2023-07-14T19:27:03.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 0x7f4e100694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 0x7f4e100694c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40864/0 (socket says 172.21.15.82:40864) 2023-07-14T19:27:03.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 -- 172.21.15.82:0/2671974303 learned_addr learned my addr 172.21.15.82:0/2671974303 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:03.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e0ffff700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e100ffd70 0x7f4e1019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e155ae700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e10100700 0x7f4e1019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 -- 172.21.15.82:0/2671974303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e10100700 msgr2=0x7f4e1019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e10100700 0x7f4e1019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 -- 172.21.15.82:0/2671974303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e100ffd70 msgr2=0x7f4e1019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e100ffd70 0x7f4e1019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.449+0000 7f4e14dad700 1 -- 172.21.15.82:0/2671974303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0095f0 con 0x7f4e100feb80 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.450+0000 7f4e14dad700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 0x7f4e100694c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400d980 tx=0x7f4e0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.450+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004020 con 0x7f4e100feb80 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.450+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0400f690 con 0x7f4e100feb80 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.450+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04003a60 con 0x7f4e100feb80 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.450+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1019da40 con 0x7f4e100feb80 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.450+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1019df00 con 0x7f4e100feb80 2023-07-14T19:27:03.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.451+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e10106070 con 0x7f4e100feb80 2023-07-14T19:27:03.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.452+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e0400fb00 con 0x7f4e100feb80 2023-07-14T19:27:03.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.453+0000 7f4e0dffb700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4e000a11d0 0x7f4e000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.453+0000 7f4e0ffff700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4e000a11d0 0x7f4e000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.453+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4e040cc450 con 0x7f4e100feb80 2023-07-14T19:27:03.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.453+0000 7f4e0ffff700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4e000a11d0 0x7f4e000a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00b5c0 tx=0x7f4dfc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:03.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.457+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4e04095c50 con 0x7f4e100feb80 2023-07-14T19:27:03.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.681+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e10060a80 con 0x7f4e000a11d0 2023-07-14T19:27:03.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.706+0000 7f4e0dffb700 1 -- 172.21.15.82:0/2671974303 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4e10060a80 con 0x7f4e000a11d0 2023-07-14T19:27:03.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4e000a11d0 msgr2=0x7f4e000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.709 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4e000a11d0 0x7f4e000a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00b5c0 tx=0x7f4dfc005960 comp rx=0 tx=0).stop 2023-07-14T19:27:03.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 msgr2=0x7f4e100694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 0x7f4e100694c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400d980 tx=0x7f4e0400dc90 comp rx=0 tx=0).stop 2023-07-14T19:27:03.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 shutdown_connections 2023-07-14T19:27:03.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4e10100700 0x7f4e1019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4e000a11d0 0x7f4e000a3680 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e100feb80 0x7f4e100694c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 --2- 172.21.15.82:0/2671974303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e100ffd70 0x7f4e1019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 >> 172.21.15.82:0/2671974303 conn(0x7f4e100fa1a0 msgr2=0x7f4e101031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:03.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 shutdown_connections 2023-07-14T19:27:03.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.708+0000 7f4e17011700 1 -- 172.21.15.82:0/2671974303 wait complete. 2023-07-14T19:27:03.721 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-14T19:27:03.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 -- 172.21.15.82:0/1366924166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000fe280 msgr2=0x7f77000fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 --2- 172.21.15.82:0/1366924166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000fe280 0x7f77000fe690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009990 tx=0x7f76f4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 -- 172.21.15.82:0/1366924166 shutdown_connections 2023-07-14T19:27:03.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 --2- 172.21.15.82:0/1366924166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77000ffe00 0x7f77001066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 --2- 172.21.15.82:0/1366924166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000fe280 0x7f77000fe690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 --2- 172.21.15.82:0/1366924166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77000ff470 0x7f77000ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 -- 172.21.15.82:0/1366924166 >> 172.21.15.82:0/1366924166 conn(0x7f77000f98a0 msgr2=0x7f77000fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:03.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 -- 172.21.15.82:0/1366924166 shutdown_connections 2023-07-14T19:27:03.848 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.846+0000 7f7706ce7700 1 -- 172.21.15.82:0/1366924166 wait complete. 2023-07-14T19:27:03.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 Processor -- start 2023-07-14T19:27:03.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 -- start start 2023-07-14T19:27:03.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77000fe280 0x7f7700195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 0x7f7700195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.850 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77000ffe00 0x7f770019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019d4d0 con 0x7f77000ff470 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019d610 con 0x7f77000fe280 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7706ce7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770019d7a0 con 0x7f77000ffe00 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 0x7f7700195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 0x7f7700195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40882/0 (socket says 172.21.15.82:40882) 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 -- 172.21.15.82:0/142075766 learned_addr learned my addr 172.21.15.82:0/142075766 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f7704a83700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77000fe280 0x7f7700195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 -- 172.21.15.82:0/142075766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77000ffe00 msgr2=0x7f770019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77000ffe00 0x7f770019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 -- 172.21.15.82:0/142075766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77000fe280 msgr2=0x7f7700195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77000fe280 0x7f7700195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.847+0000 7f76fffff700 1 -- 172.21.15.82:0/142075766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f40095f0 con 0x7f77000ff470 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.848+0000 7f76fffff700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 0x7f7700195e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00c920 tx=0x7f76ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.848+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec007b00 con 0x7f77000ff470 2023-07-14T19:27:03.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.848+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76ec00ce70 con 0x7f77000ff470 2023-07-14T19:27:03.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.848+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770019da80 con 0x7f77000ff470 2023-07-14T19:27:03.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.848+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0054e0 con 0x7f77000ff470 2023-07-14T19:27:03.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.848+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770019df90 con 0x7f77000ff470 2023-07-14T19:27:03.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.849+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7700105200 con 0x7f77000ff470 2023-07-14T19:27:03.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.850+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f76ec004750 con 0x7f77000ff470 2023-07-14T19:27:03.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.850+0000 7f76fdffb700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76f00a1150 0x7f76f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:03.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.850+0000 7f7704a83700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76f00a1150 0x7f76f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:03.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.850+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f76ec0cc320 con 0x7f77000ff470 2023-07-14T19:27:03.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.851+0000 7f7704a83700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76f00a1150 0x7f76f00a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76f400b5c0 tx=0x7f76f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:03.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:03.854+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f76ec095bc0 con 0x7f77000ff470 2023-07-14T19:27:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.079+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7700060a80 con 0x7f76f00a1150 2023-07-14T19:27:04.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.083+0000 7f76fdffb700 1 -- 172.21.15.82:0/142075766 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f7700060a80 con 0x7f76f00a1150 2023-07-14T19:27:04.084 INFO:teuthology.orchestra.run.smithi082.stdout:/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505 2023-07-14T19:27:04.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76f00a1150 msgr2=0x7f76f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76f00a1150 0x7f76f00a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76f400b5c0 tx=0x7f76f4005fd0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 msgr2=0x7f7700195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 0x7f7700195e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00c920 tx=0x7f76ec00cc30 comp rx=0 tx=0).stop 2023-07-14T19:27:04.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 shutdown_connections 2023-07-14T19:27:04.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f77000ffe00 0x7f770019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77000fe280 0x7f7700195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76f00a1150 0x7f76f00a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 --2- 172.21.15.82:0/142075766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77000ff470 0x7f7700195e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 >> 172.21.15.82:0/142075766 conn(0x7f77000f98a0 msgr2=0x7f77000fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:04.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 shutdown_connections 2023-07-14T19:27:04.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.085+0000 7f7706ce7700 1 -- 172.21.15.82:0/142075766 wait complete. 2023-07-14T19:27:04.097 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:03 smithi082 ceph-mon[127461]: pgmap v250: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 12 op/s 2023-07-14T19:27:04.097 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:03 smithi082 ceph-mon[127461]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:04.099 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:27:04.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:03 smithi133 ceph-mon[128894]: pgmap v250: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 12 op/s 2023-07-14T19:27:04.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:03 smithi133 ceph-mon[128894]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:04.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 -- 172.21.15.82:0/3216577066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54280ffcf0 msgr2=0x7f5428100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 --2- 172.21.15.82:0/3216577066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54280ffcf0 0x7f5428100140 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f541c009990 tx=0x7f541c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 -- 172.21.15.82:0/3216577066 shutdown_connections 2023-07-14T19:27:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 --2- 172.21.15.82:0/3216577066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5428100680 0x7f5428106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 --2- 172.21.15.82:0/3216577066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54280feb00 0x7f54280fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 --2- 172.21.15.82:0/3216577066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54280ffcf0 0x7f5428100140 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 -- 172.21.15.82:0/3216577066 >> 172.21.15.82:0/3216577066 conn(0x7f54280fa100 msgr2=0x7f54280fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 -- 172.21.15.82:0/3216577066 shutdown_connections 2023-07-14T19:27:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.220+0000 7f54306d6700 1 -- 172.21.15.82:0/3216577066 wait complete. 2023-07-14T19:27:04.223 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 Processor -- start 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 -- start start 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54280feb00 0x7f5428102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280ffcf0 0x7f54281010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 0x7f5428101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428101db0 con 0x7f5428100680 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428101f20 con 0x7f54280feb00 2023-07-14T19:27:04.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542819f940 con 0x7f54280ffcf0 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 0x7f5428101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 0x7f5428101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40898/0 (socket says 172.21.15.82:40898) 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 -- 172.21.15.82:0/114393644 learned_addr learned my addr 172.21.15.82:0/114393644 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542e472700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54280feb00 0x7f5428102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542dc71700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280ffcf0 0x7f54281010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 -- 172.21.15.82:0/114393644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280ffcf0 msgr2=0x7f54281010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280ffcf0 0x7f54281010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 -- 172.21.15.82:0/114393644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54280feb00 msgr2=0x7f5428102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54280feb00 0x7f5428102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 -- 172.21.15.82:0/114393644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0095f0 con 0x7f5428100680 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f542ec73700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 0x7f5428101610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5420007ee0 tx=0x7f542000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420017070 con 0x7f5428100680 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f542000f040 con 0x7f5428100680 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542819fc20 con 0x7f5428100680 2023-07-14T19:27:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.221+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420005740 con 0x7f5428100680 2023-07-14T19:27:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.222+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54281a00e0 con 0x7f5428100680 2023-07-14T19:27:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.223+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542810a2d0 con 0x7f5428100680 2023-07-14T19:27:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.224+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f542000dc60 con 0x7f5428100680 2023-07-14T19:27:04.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.224+0000 7f541b7fe700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f54140a11a0 0x7f54140a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.224+0000 7f542e472700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f54140a11a0 0x7f54140a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.224+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f54200cbfd0 con 0x7f5428100680 2023-07-14T19:27:04.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.225+0000 7f542e472700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f54140a11a0 0x7f54140a3650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f54280ffb50 tx=0x7f5424009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:04.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.228+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5420095870 con 0x7f5428100680 2023-07-14T19:27:04.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:03 smithi161 ceph-mon[129184]: pgmap v250: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 12 op/s 2023-07-14T19:27:04.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:03 smithi161 ceph-mon[129184]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:04.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.460+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54281a0690 con 0x7f5428100680 2023-07-14T19:27:04.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.461+0000 7f541b7fe700 1 -- 172.21.15.82:0/114393644 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f5420098e90 con 0x7f5428100680 2023-07-14T19:27:04.461 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:04.462 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:27:02.519960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:27:04.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f54140a11a0 msgr2=0x7f54140a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f54140a11a0 0x7f54140a3650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f54280ffb50 tx=0x7f5424009260 comp rx=0 tx=0).stop 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 msgr2=0x7f5428101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 0x7f5428101610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5420007ee0 tx=0x7f542000d3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 shutdown_connections 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f54140a11a0 0x7f54140a3650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5428100680 0x7f5428101610 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54280feb00 0x7f5428102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 --2- 172.21.15.82:0/114393644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54280ffcf0 0x7f54281010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 >> 172.21.15.82:0/114393644 conn(0x7f54280fa100 msgr2=0x7f54280677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:04.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 shutdown_connections 2023-07-14T19:27:04.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.463+0000 7f54306d6700 1 -- 172.21.15.82:0/114393644 wait complete. 2023-07-14T19:27:04.465 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:27:04.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:27:04.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 -- 172.21.15.82:0/1467079997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 msgr2=0x7f9348106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 --2- 172.21.15.82:0/1467079997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f9348106f60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f934000b0a0 tx=0x7f934000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 -- 172.21.15.82:0/1467079997 shutdown_connections 2023-07-14T19:27:04.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 --2- 172.21.15.82:0/1467079997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f9348106f60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 --2- 172.21.15.82:0/1467079997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93480feb20 0x7f93480fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 --2- 172.21.15.82:0/1467079997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480ffd10 0x7f9348100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 -- 172.21.15.82:0/1467079997 >> 172.21.15.82:0/1467079997 conn(0x7f93480fa100 msgr2=0x7f93480fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 -- 172.21.15.82:0/1467079997 shutdown_connections 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.598+0000 7f934eb44700 1 -- 172.21.15.82:0/1467079997 wait complete. 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 Processor -- start 2023-07-14T19:27:04.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 -- start start 2023-07-14T19:27:04.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480feb20 0x7f9348194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93480ffd10 0x7f9348194e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f934819bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934819c4c0 con 0x7f93481006a0 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934819c600 con 0x7f93480ffd10 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934eb44700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934819c790 con 0x7f93480feb20 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f934819bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f934819bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40908/0 (socket says 172.21.15.82:40908) 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 -- 172.21.15.82:0/2808613849 learned_addr learned my addr 172.21.15.82:0/2808613849 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:04.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f9347fff700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93480ffd10 0x7f9348194e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934c8e0700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480feb20 0x7f9348194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 -- 172.21.15.82:0/2808613849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480feb20 msgr2=0x7f9348194940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480feb20 0x7f9348194940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 -- 172.21.15.82:0/2808613849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93480ffd10 msgr2=0x7f9348194e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93480ffd10 0x7f9348194e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.599+0000 7f934d0e1700 1 -- 172.21.15.82:0/2808613849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9340009d00 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.600+0000 7f934d0e1700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f934819bef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f934000d010 tx=0x7f934000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.600+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934000e040 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.600+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9340003e10 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.600+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934819ca10 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.600+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400052a0 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.600+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934819ce30 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.601+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934810a350 con 0x7f93481006a0 2023-07-14T19:27:04.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.602+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9340005400 con 0x7f93481006a0 2023-07-14T19:27:04.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.602+0000 7f9345ffb700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93380a11a0 0x7f93380a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.603+0000 7f934c8e0700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93380a11a0 0x7f93380a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.603+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93400cdca0 con 0x7f93481006a0 2023-07-14T19:27:04.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.603+0000 7f934c8e0700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93380a11a0 0x7f93380a3650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93480ffb70 tx=0x7f933c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:04.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.606+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f93400975d0 con 0x7f93481006a0 2023-07-14T19:27:04.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.790+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f93400979a0 con 0x7f93481006a0 2023-07-14T19:27:04.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.806+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9348105920 con 0x7f93481006a0 2023-07-14T19:27:04.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.808+0000 7f9345ffb700 1 -- 172.21.15.82:0/2808613849 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f934005afb0 con 0x7f93481006a0 2023-07-14T19:27:04.809 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:04.809 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:04.769027+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93380a11a0 msgr2=0x7f93380a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93380a11a0 0x7f93380a3650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93480ffb70 tx=0x7f933c009260 comp rx=0 tx=0).stop 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 msgr2=0x7f934819bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f934819bef0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f934000d010 tx=0x7f934000bc80 comp rx=0 tx=0).stop 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 shutdown_connections 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f93380a11a0 0x7f93380a3650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93481006a0 0x7f934819bef0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480feb20 0x7f9348194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 --2- 172.21.15.82:0/2808613849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93480ffd10 0x7f9348194e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 >> 172.21.15.82:0/2808613849 conn(0x7f93480fa100 msgr2=0x7f93480fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:04.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 shutdown_connections 2023-07-14T19:27:04.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.810+0000 7f934eb44700 1 -- 172.21.15.82:0/2808613849 wait complete. 2023-07-14T19:27:04.824 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-14T19:27:04.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T19:27:04.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 -- 172.21.15.82:0/4268588751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c0feb00 msgr2=0x7f9b2c0fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 --2- 172.21.15.82:0/4268588751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c0feb00 0x7f9b2c0fef10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009990 tx=0x7f9b1c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 -- 172.21.15.82:0/4268588751 shutdown_connections 2023-07-14T19:27:04.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 --2- 172.21.15.82:0/4268588751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2c100680 0x7f9b2c106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 --2- 172.21.15.82:0/4268588751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c0feb00 0x7f9b2c0fef10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 --2- 172.21.15.82:0/4268588751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b2c0ffcf0 0x7f9b2c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 -- 172.21.15.82:0/4268588751 >> 172.21.15.82:0/4268588751 conn(0x7f9b2c0fa100 msgr2=0x7f9b2c0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:04.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 -- 172.21.15.82:0/4268588751 shutdown_connections 2023-07-14T19:27:04.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.945+0000 7f9b308cb700 1 -- 172.21.15.82:0/4268588751 wait complete. 2023-07-14T19:27:04.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 Processor -- start 2023-07-14T19:27:04.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 -- start start 2023-07-14T19:27:04.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b2c0feb00 0x7f9b2c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2c0ffcf0 0x7f9b2c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 0x7f9b2c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c19d460 con 0x7f9b2c100680 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c19d5a0 con 0x7f9b2c0feb00 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b308cb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c19d730 con 0x7f9b2c0ffcf0 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 0x7f9b2c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 0x7f9b2c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40922/0 (socket says 172.21.15.82:40922) 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 -- 172.21.15.82:0/1063064920 learned_addr learned my addr 172.21.15.82:0/1063064920 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b29d9b700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2c0ffcf0 0x7f9b2c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2a59c700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b2c0feb00 0x7f9b2c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 -- 172.21.15.82:0/1063064920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2c0ffcf0 msgr2=0x7f9b2c195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2c0ffcf0 0x7f9b2c195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 -- 172.21.15.82:0/1063064920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b2c0feb00 msgr2=0x7f9b2c1958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:04.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b2c0feb00 0x7f9b2c1958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:04.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 -- 172.21.15.82:0/1063064920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0095f0 con 0x7f9b2c100680 2023-07-14T19:27:04.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.946+0000 7f9b2ad9d700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 0x7f9b2c19ce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400b970 tx=0x7f9b2400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.947+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400c930 con 0x7f9b2c100680 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.947+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b24014440 con 0x7f9b2c100680 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.947+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b240075d0 con 0x7f9b2c100680 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.947+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c19da10 con 0x7f9b2c100680 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.947+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c19df50 con 0x7f9b2c100680 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.948+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c106b40 con 0x7f9b2c100680 2023-07-14T19:27:04.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.948+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9b240145b0 con 0x7f9b2c100680 2023-07-14T19:27:04.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.949+0000 7f9b1b7fe700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9b140a12a0 0x7f9b140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:04.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.949+0000 7f9b2a59c700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9b140a12a0 0x7f9b140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:04.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.949+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f9b24006e40 con 0x7f9b2c100680 2023-07-14T19:27:04.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.950+0000 7f9b2a59c700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9b140a12a0 0x7f9b140a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006010 tx=0x7f9b1c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:04.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:04.954+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9b2404e780 con 0x7f9b2c100680 2023-07-14T19:27:05.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.137+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b2c060a80 con 0x7f9b140a12a0 2023-07-14T19:27:05.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.148+0000 7f9b1b7fe700 1 -- 172.21.15.82:0/1063064920 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335958 (secure 0 0 0) 0x7f9b2c060a80 con 0x7f9b140a12a0 2023-07-14T19:27:05.155 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:05.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9b140a12a0 msgr2=0x7f9b140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9b140a12a0 0x7f9b140a3750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c006010 tx=0x7f9b1c00b540 comp rx=0 tx=0).stop 2023-07-14T19:27:05.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 msgr2=0x7f9b2c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.158 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 0x7f9b2c19ce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400b970 tx=0x7f9b2400bc80 comp rx=0 tx=0).stop 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 shutdown_connections 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f9b140a12a0 0x7f9b140a3750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c100680 0x7f9b2c19ce90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b2c0feb00 0x7f9b2c1958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 --2- 172.21.15.82:0/1063064920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b2c0ffcf0 0x7f9b2c195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 >> 172.21.15.82:0/1063064920 conn(0x7f9b2c0fa100 msgr2=0x7f9b2c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:05.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 shutdown_connections 2023-07-14T19:27:05.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.151+0000 7f9b308cb700 1 -- 172.21.15.82:0/1063064920 wait complete. 2023-07-14T19:27:05.160 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:27:05.163 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.163 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.163 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.163 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/114393644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:05.163 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T19:27:05.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-14T19:27:05.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T19:27:05.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-07-14T19:27:05.164 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:04 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-07-14T19:27:05.165 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-07-14T19:27:04.778882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594872,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":141,"num_read_kb":124,"num_write":282,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334660,"kb_used_data":10420,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124607164,"statfs":{"total":1151940427776,"available":1151597735936,"internally_reserved":0,"allocated":10670080,"data_stored":6751684,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4504,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":101,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.474057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307225+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:26:58.307225+0000","last_peered":"2023-07-14T19:26:58.307225+0000","last_clean":"2023-07-14T19:26:58.307225+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:26:58.307225+0000","last_undegraded":"2023-07-14T19:26:58.307225+0000","last_fullsized":"2023-07-14T19:26:58.307225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:11:53.276829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022204+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:26:59.022204+0000","last_peered":"2023-07-14T19:26:59.022204+0000","last_clean":"2023-07-14T19:26:59.022204+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:26:59.022204+0000","last_undegraded":"2023-07-14T19:26:59.022204+0000","last_fullsized":"2023-07-14T19:26:59.022204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:52:37.455002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021492+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:26:59.021492+0000","last_peered":"2023-07-14T19:26:59.021492+0000","last_clean":"2023-07-14T19:26:59.021492+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:26:59.021492+0000","last_undegraded":"2023-07-14T19:26:59.021492+0000","last_fullsized":"2023-07-14T19:26:59.021492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:04:17.827893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022921+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:26:59.022921+0000","last_peered":"2023-07-14T19:26:59.022921+0000","last_clean":"2023-07-14T19:26:59.022921+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:26:59.022921+0000","last_undegraded":"2023-07-14T19:26:59.022921+0000","last_fullsized":"2023-07-14T19:26:59.022921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:42.948302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725619+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:26:58.725619+0000","last_peered":"2023-07-14T19:26:58.725619+0000","last_clean":"2023-07-14T19:26:58.725619+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:26:58.725619+0000","last_undegraded":"2023-07-14T19:26:58.725619+0000","last_fullsized":"2023-07-14T19:26:58.725619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:56.134158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308235+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:26:58.308235+0000","last_peered":"2023-07-14T19:26:58.308235+0000","last_clean":"2023-07-14T19:26:58.308235+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:26:58.308235+0000","last_undegraded":"2023-07-14T19:26:58.308235+0000","last_fullsized":"2023-07-14T19:26:58.308235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:55.795598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479553+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:26:58.479553+0000","last_peered":"2023-07-14T19:26:58.479553+0000","last_clean":"2023-07-14T19:26:58.479553+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:26:58.479553+0000","last_undegraded":"2023-07-14T19:26:58.479553+0000","last_fullsized":"2023-07-14T19:26:58.479553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:03.769700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.306961+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:26:58.306961+0000","last_peered":"2023-07-14T19:26:58.306961+0000","last_clean":"2023-07-14T19:26:58.306961+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:26:58.306961+0000","last_undegraded":"2023-07-14T19:26:58.306961+0000","last_fullsized":"2023-07-14T19:26:58.306961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:39:55.428976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022255+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:26:59.022255+0000","last_peered":"2023-07-14T19:26:59.022255+0000","last_clean":"2023-07-14T19:26:59.022255+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:26:59.022255+0000","last_undegraded":"2023-07-14T19:26:59.022255+0000","last_fullsized":"2023-07-14T19:26:59.022255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:43:45.852784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308658+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:26:58.308658+0000","last_peered":"2023-07-14T19:26:58.308658+0000","last_clean":"2023-07-14T19:26:58.308658+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:26:58.308658+0000","last_undegraded":"2023-07-14T19:26:58.308658+0000","last_fullsized":"2023-07-14T19:26:58.308658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:54:35.306168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479735+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:26:58.479735+0000","last_peered":"2023-07-14T19:26:58.479735+0000","last_clean":"2023-07-14T19:26:58.479735+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:26:58.479735+0000","last_undegraded":"2023-07-14T19:26:58.479735+0000","last_fullsized":"2023-07-14T19:26:58.479735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:26:26.037572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022843+0000","last_change":"2023-07-14T19:26:57.324895+0000","last_active":"2023-07-14T19:26:59.022843+0000","last_peered":"2023-07-14T19:26:59.022843+0000","last_clean":"2023-07-14T19:26:59.022843+0000","last_became_active":"2023-07-14T19:26:57.324344+0000","last_became_peered":"2023-07-14T19:26:57.324344+0000","last_unstale":"2023-07-14T19:26:59.022843+0000","last_undegraded":"2023-07-14T19:26:59.022843+0000","last_fullsized":"2023-07-14T19:26:59.022843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:36:34.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022126+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:26:59.022126+0000","last_peered":"2023-07-14T19:26:59.022126+0000","last_clean":"2023-07-14T19:26:59.022126+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:26:59.022126+0000","last_undegraded":"2023-07-14T19:26:59.022126+0000","last_fullsized":"2023-07-14T19:26:59.022126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:17:01.906908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021558+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:26:59.021558+0000","last_peered":"2023-07-14T19:26:59.021558+0000","last_clean":"2023-07-14T19:26:59.021558+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:26:59.021558+0000","last_undegraded":"2023-07-14T19:26:59.021558+0000","last_fullsized":"2023-07-14T19:26:59.021558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:17.055224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021637+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:26:59.021637+0000","last_peered":"2023-07-14T19:26:59.021637+0000","last_clean":"2023-07-14T19:26:59.021637+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:26:59.021637+0000","last_undegraded":"2023-07-14T19:26:59.021637+0000","last_fullsized":"2023-07-14T19:26:59.021637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:26.971168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021570+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:26:59.021570+0000","last_peered":"2023-07-14T19:26:59.021570+0000","last_clean":"2023-07-14T19:26:59.021570+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:26:59.021570+0000","last_undegraded":"2023-07-14T19:26:59.021570+0000","last_fullsized":"2023-07-14T19:26:59.021570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:38.758691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307380+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:26:58.307380+0000","last_peered":"2023-07-14T19:26:58.307380+0000","last_clean":"2023-07-14T19:26:58.307380+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:26:58.307380+0000","last_undegraded":"2023-07-14T19:26:58.307380+0000","last_fullsized":"2023-07-14T19:26:58.307380+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:21:48.872090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021520+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:26:59.021520+0000","last_peered":"2023-07-14T19:26:59.021520+0000","last_clean":"2023-07-14T19:26:59.021520+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:26:59.021520+0000","last_undegraded":"2023-07-14T19:26:59.021520+0000","last_fullsized":"2023-07-14T19:26:59.021520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:09:32.103654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022152+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:26:59.022152+0000","last_peered":"2023-07-14T19:26:59.022152+0000","last_clean":"2023-07-14T19:26:59.022152+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:26:59.022152+0000","last_undegraded":"2023-07-14T19:26:59.022152+0000","last_fullsized":"2023-07-14T19:26:59.022152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:54:36.812775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022120+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:26:59.022120+0000","last_peered":"2023-07-14T19:26:59.022120+0000","last_clean":"2023-07-14T19:26:59.022120+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:26:59.022120+0000","last_undegraded":"2023-07-14T19:26:59.022120+0000","last_fullsized":"2023-07-14T19:26:59.022120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:17.414571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307265+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:26:58.307265+0000","last_peered":"2023-07-14T19:26:58.307265+0000","last_clean":"2023-07-14T19:26:58.307265+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:26:58.307265+0000","last_undegraded":"2023-07-14T19:26:58.307265+0000","last_fullsized":"2023-07-14T19:26:58.307265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:46.472799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.717376+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:26:58.717376+0000","last_peered":"2023-07-14T19:26:58.717376+0000","last_clean":"2023-07-14T19:26:58.717376+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:26:58.717376+0000","last_undegraded":"2023-07-14T19:26:58.717376+0000","last_fullsized":"2023-07-14T19:26:58.717376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:17.440990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021987+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:26:59.021987+0000","last_peered":"2023-07-14T19:26:59.021987+0000","last_clean":"2023-07-14T19:26:59.021987+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:26:59.021987+0000","last_undegraded":"2023-07-14T19:26:59.021987+0000","last_fullsized":"2023-07-14T19:26:59.021987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:56:02.835550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.779342+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:01.779342+0000","last_peered":"2023-07-14T19:27:01.779342+0000","last_clean":"2023-07-14T19:27:01.779342+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:01.779342+0000","last_undegraded":"2023-07-14T19:27:01.779342+0000","last_fullsized":"2023-07-14T19:27:01.779342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:29:40.372439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021751+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:26:59.021751+0000","last_peered":"2023-07-14T19:26:59.021751+0000","last_clean":"2023-07-14T19:26:59.021751+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:26:59.021751+0000","last_undegraded":"2023-07-14T19:26:59.021751+0000","last_fullsized":"2023-07-14T19:26:59.021751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:02:32.436754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725830+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:26:58.725830+0000","last_peered":"2023-07-14T19:26:58.725830+0000","last_clean":"2023-07-14T19:26:58.725830+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:26:58.725830+0000","last_undegraded":"2023-07-14T19:26:58.725830+0000","last_fullsized":"2023-07-14T19:26:58.725830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:52:15.739288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022396+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:26:59.022396+0000","last_peered":"2023-07-14T19:26:59.022396+0000","last_clean":"2023-07-14T19:26:59.022396+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:26:59.022396+0000","last_undegraded":"2023-07-14T19:26:59.022396+0000","last_fullsized":"2023-07-14T19:26:59.022396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:56:29.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.777934+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:27:00.777934+0000","last_peered":"2023-07-14T19:27:00.777934+0000","last_clean":"2023-07-14T19:27:00.777934+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:27:00.777934+0000","last_undegraded":"2023-07-14T19:27:00.777934+0000","last_fullsized":"2023-07-14T19:27:00.777934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:15:04.045080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725644+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:26:58.725644+0000","last_peered":"2023-07-14T19:26:58.725644+0000","last_clean":"2023-07-14T19:26:58.725644+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:26:58.725644+0000","last_undegraded":"2023-07-14T19:26:58.725644+0000","last_fullsized":"2023-07-14T19:26:58.725644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:14.106188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308121+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:26:58.308121+0000","last_peered":"2023-07-14T19:26:58.308121+0000","last_clean":"2023-07-14T19:26:58.308121+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:26:58.308121+0000","last_undegraded":"2023-07-14T19:26:58.308121+0000","last_fullsized":"2023-07-14T19:26:58.308121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:57:10.165499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022669+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:26:59.022669+0000","last_peered":"2023-07-14T19:26:59.022669+0000","last_clean":"2023-07-14T19:26:59.022669+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:26:59.022669+0000","last_undegraded":"2023-07-14T19:26:59.022669+0000","last_fullsized":"2023-07-14T19:26:59.022669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:58:17.122017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.781505+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:01.781505+0000","last_peered":"2023-07-14T19:27:01.781505+0000","last_clean":"2023-07-14T19:27:01.781505+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:01.781505+0000","last_undegraded":"2023-07-14T19:27:01.781505+0000","last_fullsized":"2023-07-14T19:27:01.781505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:24.062279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307652+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:26:58.307652+0000","last_peered":"2023-07-14T19:26:58.307652+0000","last_clean":"2023-07-14T19:26:58.307652+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:26:58.307652+0000","last_undegraded":"2023-07-14T19:26:58.307652+0000","last_fullsized":"2023-07-14T19:26:58.307652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:33.497311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022394+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:26:59.022394+0000","last_peered":"2023-07-14T19:26:59.022394+0000","last_clean":"2023-07-14T19:26:59.022394+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:26:59.022394+0000","last_undegraded":"2023-07-14T19:26:59.022394+0000","last_fullsized":"2023-07-14T19:26:59.022394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:02.102706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021535+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:26:59.021535+0000","last_peered":"2023-07-14T19:26:59.021535+0000","last_clean":"2023-07-14T19:26:59.021535+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:26:59.021535+0000","last_undegraded":"2023-07-14T19:26:59.021535+0000","last_fullsized":"2023-07-14T19:26:59.021535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:51:08.593239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.778707+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:27:00.778707+0000","last_peered":"2023-07-14T19:27:00.778707+0000","last_clean":"2023-07-14T19:27:00.778707+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:27:00.778707+0000","last_undegraded":"2023-07-14T19:27:00.778707+0000","last_fullsized":"2023-07-14T19:27:00.778707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:13.775871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022138+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:26:59.022138+0000","last_peered":"2023-07-14T19:26:59.022138+0000","last_clean":"2023-07-14T19:26:59.022138+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:26:59.022138+0000","last_undegraded":"2023-07-14T19:26:59.022138+0000","last_fullsized":"2023-07-14T19:26:59.022138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:15:26.311344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.777334+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:00.777334+0000","last_peered":"2023-07-14T19:27:00.777334+0000","last_clean":"2023-07-14T19:27:00.777334+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:00.777334+0000","last_undegraded":"2023-07-14T19:27:00.777334+0000","last_fullsized":"2023-07-14T19:27:00.777334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:23:41.881340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021853+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:26:59.021853+0000","last_peered":"2023-07-14T19:26:59.021853+0000","last_clean":"2023-07-14T19:26:59.021853+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:26:59.021853+0000","last_undegraded":"2023-07-14T19:26:59.021853+0000","last_fullsized":"2023-07-14T19:26:59.021853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:34.073631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022610+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:26:59.022610+0000","last_peered":"2023-07-14T19:26:59.022610+0000","last_clean":"2023-07-14T19:26:59.022610+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:26:59.022610+0000","last_undegraded":"2023-07-14T19:26:59.022610+0000","last_fullsized":"2023-07-14T19:26:59.022610+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:15.187668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021910+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:26:59.021910+0000","last_peered":"2023-07-14T19:26:59.021910+0000","last_clean":"2023-07-14T19:26:59.021910+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:26:59.021910+0000","last_undegraded":"2023-07-14T19:26:59.021910+0000","last_fullsized":"2023-07-14T19:26:59.021910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:12:50.801527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.779373+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:27:00.779373+0000","last_peered":"2023-07-14T19:27:00.779373+0000","last_clean":"2023-07-14T19:27:00.779373+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:27:00.779373+0000","last_undegraded":"2023-07-14T19:27:00.779373+0000","last_fullsized":"2023-07-14T19:27:00.779373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:12.785323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725819+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:26:58.725819+0000","last_peered":"2023-07-14T19:26:58.725819+0000","last_clean":"2023-07-14T19:26:58.725819+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:26:58.725819+0000","last_undegraded":"2023-07-14T19:26:58.725819+0000","last_fullsized":"2023-07-14T19:26:58.725819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:38.092509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.780074+0000","last_change":"2023-07-14T19:26:56.500601+0000","last_active":"2023-07-14T19:27:01.780074+0000","last_peered":"2023-07-14T19:27:01.780074+0000","last_clean":"2023-07-14T19:27:01.780074+0000","last_became_active":"2023-07-14T19:26:56.500296+0000","last_became_peered":"2023-07-14T19:26:56.500296+0000","last_unstale":"2023-07-14T19:27:01.780074+0000","last_undegraded":"2023-07-14T19:27:01.780074+0000","last_fullsized":"2023-07-14T19:27:01.780074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:54:33.136846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022264+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:26:59.022264+0000","last_peered":"2023-07-14T19:26:59.022264+0000","last_clean":"2023-07-14T19:26:59.022264+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:26:59.022264+0000","last_undegraded":"2023-07-14T19:26:59.022264+0000","last_fullsized":"2023-07-14T19:26:59.022264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:02:31.957284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.776790+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:00.776790+0000","last_peered":"2023-07-14T19:27:00.776790+0000","last_clean":"2023-07-14T19:27:00.776790+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:00.776790+0000","last_undegraded":"2023-07-14T19:27:00.776790+0000","last_fullsized":"2023-07-14T19:27:00.776790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:34.301368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479787+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:26:58.479787+0000","last_peered":"2023-07-14T19:26:58.479787+0000","last_clean":"2023-07-14T19:26:58.479787+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:26:58.479787+0000","last_undegraded":"2023-07-14T19:26:58.479787+0000","last_fullsized":"2023-07-14T19:26:58.479787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:05:40.828316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022639+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:26:59.022639+0000","last_peered":"2023-07-14T19:26:59.022639+0000","last_clean":"2023-07-14T19:26:59.022639+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:26:59.022639+0000","last_undegraded":"2023-07-14T19:26:59.022639+0000","last_fullsized":"2023-07-14T19:26:59.022639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:25.824522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479466+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:26:58.479466+0000","last_peered":"2023-07-14T19:26:58.479466+0000","last_clean":"2023-07-14T19:26:58.479466+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:26:58.479466+0000","last_undegraded":"2023-07-14T19:26:58.479466+0000","last_fullsized":"2023-07-14T19:26:58.479466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:10:44.012832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.781194+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:01.781194+0000","last_peered":"2023-07-14T19:27:01.781194+0000","last_clean":"2023-07-14T19:27:01.781194+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:01.781194+0000","last_undegraded":"2023-07-14T19:27:01.781194+0000","last_fullsized":"2023-07-14T19:27:01.781194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:07:24.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022500+0000","last_change":"2023-07-14T19:26:57.323495+0000","last_active":"2023-07-14T19:26:59.022500+0000","last_peered":"2023-07-14T19:26:59.022500+0000","last_clean":"2023-07-14T19:26:59.022500+0000","last_became_active":"2023-07-14T19:26:57.323199+0000","last_became_peered":"2023-07-14T19:26:57.323199+0000","last_unstale":"2023-07-14T19:26:59.022500+0000","last_undegraded":"2023-07-14T19:26:59.022500+0000","last_fullsized":"2023-07-14T19:26:59.022500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:11:39.678080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725961+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:26:58.725961+0000","last_peered":"2023-07-14T19:26:58.725961+0000","last_clean":"2023-07-14T19:26:58.725961+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:26:58.725961+0000","last_undegraded":"2023-07-14T19:26:58.725961+0000","last_fullsized":"2023-07-14T19:26:58.725961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:32:35.481135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307225+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:26:58.307225+0000","last_peered":"2023-07-14T19:26:58.307225+0000","last_clean":"2023-07-14T19:26:58.307225+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:26:58.307225+0000","last_undegraded":"2023-07-14T19:26:58.307225+0000","last_fullsized":"2023-07-14T19:26:58.307225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:59.627293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021920+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:26:59.021920+0000","last_peered":"2023-07-14T19:26:59.021920+0000","last_clean":"2023-07-14T19:26:59.021920+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:26:59.021920+0000","last_undegraded":"2023-07-14T19:26:59.021920+0000","last_fullsized":"2023-07-14T19:26:59.021920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:48.402550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308578+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:26:58.308578+0000","last_peered":"2023-07-14T19:26:58.308578+0000","last_clean":"2023-07-14T19:26:58.308578+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:26:58.308578+0000","last_undegraded":"2023-07-14T19:26:58.308578+0000","last_fullsized":"2023-07-14T19:26:58.308578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:59:10.880240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479879+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:26:58.479879+0000","last_peered":"2023-07-14T19:26:58.479879+0000","last_clean":"2023-07-14T19:26:58.479879+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:26:58.479879+0000","last_undegraded":"2023-07-14T19:26:58.479879+0000","last_fullsized":"2023-07-14T19:26:58.479879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:33:39.197230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308501+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:26:58.308501+0000","last_peered":"2023-07-14T19:26:58.308501+0000","last_clean":"2023-07-14T19:26:58.308501+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:26:58.308501+0000","last_undegraded":"2023-07-14T19:26:58.308501+0000","last_fullsized":"2023-07-14T19:26:58.308501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:20:26.973280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022680+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:26:59.022680+0000","last_peered":"2023-07-14T19:26:59.022680+0000","last_clean":"2023-07-14T19:26:59.022680+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:26:59.022680+0000","last_undegraded":"2023-07-14T19:26:59.022680+0000","last_fullsized":"2023-07-14T19:26:59.022680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:31:01.985023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.780434+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:27:01.780434+0000","last_peered":"2023-07-14T19:27:01.780434+0000","last_clean":"2023-07-14T19:27:01.780434+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:27:01.780434+0000","last_undegraded":"2023-07-14T19:27:01.780434+0000","last_fullsized":"2023-07-14T19:27:01.780434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:42:58.103911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307562+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:26:58.307562+0000","last_peered":"2023-07-14T19:26:58.307562+0000","last_clean":"2023-07-14T19:26:58.307562+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:26:58.307562+0000","last_undegraded":"2023-07-14T19:26:58.307562+0000","last_fullsized":"2023-07-14T19:26:58.307562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:02:19.231493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022845+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:26:59.022845+0000","last_peered":"2023-07-14T19:26:59.022845+0000","last_clean":"2023-07-14T19:26:59.022845+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:26:59.022845+0000","last_undegraded":"2023-07-14T19:26:59.022845+0000","last_fullsized":"2023-07-14T19:26:59.022845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:05:27.875468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":339,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479508+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:26:58.479508+0000","last_peered":"2023-07-14T19:26:58.479508+0000","last_clean":"2023-07-14T19:26:58.479508+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:26:58.479508+0000","last_undegraded":"2023-07-14T19:26:58.479508+0000","last_fullsized":"2023-07-14T19:26:58.479508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:56.740236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021287+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:26:59.021287+0000","last_peered":"2023-07-14T19:26:59.021287+0000","last_clean":"2023-07-14T19:26:59.021287+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:26:59.021287+0000","last_undegraded":"2023-07-14T19:26:59.021287+0000","last_fullsized":"2023-07-14T19:26:59.021287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:41:18.800844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.779712+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:01.779712+0000","last_peered":"2023-07-14T19:27:01.779712+0000","last_clean":"2023-07-14T19:27:01.779712+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:01.779712+0000","last_undegraded":"2023-07-14T19:27:01.779712+0000","last_fullsized":"2023-07-14T19:27:01.779712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:03:59.089182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.776988+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:00.776988+0000","last_peered":"2023-07-14T19:27:00.776988+0000","last_clean":"2023-07-14T19:27:00.776988+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:00.776988+0000","last_undegraded":"2023-07-14T19:27:00.776988+0000","last_fullsized":"2023-07-14T19:27:00.776988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:58:01.879837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022144+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:26:59.022144+0000","last_peered":"2023-07-14T19:26:59.022144+0000","last_clean":"2023-07-14T19:26:59.022144+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:26:59.022144+0000","last_undegraded":"2023-07-14T19:26:59.022144+0000","last_fullsized":"2023-07-14T19:26:59.022144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:25:33.109783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022034+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:26:59.022034+0000","last_peered":"2023-07-14T19:26:59.022034+0000","last_clean":"2023-07-14T19:26:59.022034+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:26:59.022034+0000","last_undegraded":"2023-07-14T19:26:59.022034+0000","last_fullsized":"2023-07-14T19:26:59.022034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:17.438048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307289+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:26:58.307289+0000","last_peered":"2023-07-14T19:26:58.307289+0000","last_clean":"2023-07-14T19:26:58.307289+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:26:58.307289+0000","last_undegraded":"2023-07-14T19:26:58.307289+0000","last_fullsized":"2023-07-14T19:26:58.307289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:17.016308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.023007+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:26:59.023007+0000","last_peered":"2023-07-14T19:26:59.023007+0000","last_clean":"2023-07-14T19:26:59.023007+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:26:59.023007+0000","last_undegraded":"2023-07-14T19:26:59.023007+0000","last_fullsized":"2023-07-14T19:26:59.023007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:04.731889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022327+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:26:59.022327+0000","last_peered":"2023-07-14T19:26:59.022327+0000","last_clean":"2023-07-14T19:26:59.022327+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:26:59.022327+0000","last_undegraded":"2023-07-14T19:26:59.022327+0000","last_fullsized":"2023-07-14T19:26:59.022327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:45.416568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.775491+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:00.775491+0000","last_peered":"2023-07-14T19:27:00.775491+0000","last_clean":"2023-07-14T19:27:00.775491+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:00.775491+0000","last_undegraded":"2023-07-14T19:27:00.775491+0000","last_fullsized":"2023-07-14T19:27:00.775491+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:13:29.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725512+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:26:58.725512+0000","last_peered":"2023-07-14T19:26:58.725512+0000","last_clean":"2023-07-14T19:26:58.725512+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:26:58.725512+0000","last_undegraded":"2023-07-14T19:26:58.725512+0000","last_fullsized":"2023-07-14T19:26:58.725512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:35.106302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.776125+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:00.776125+0000","last_peered":"2023-07-14T19:27:00.776125+0000","last_clean":"2023-07-14T19:27:00.776125+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:00.776125+0000","last_undegraded":"2023-07-14T19:27:00.776125+0000","last_fullsized":"2023-07-14T19:27:00.776125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:20:19.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307014+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:26:58.307014+0000","last_peered":"2023-07-14T19:26:58.307014+0000","last_clean":"2023-07-14T19:26:58.307014+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:26:58.307014+0000","last_undegraded":"2023-07-14T19:26:58.307014+0000","last_fullsized":"2023-07-14T19:26:58.307014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:53.548772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022015+0000","last_change":"2023-07-14T19:26:56.499194+0000","last_active":"2023-07-14T19:26:59.022015+0000","last_peered":"2023-07-14T19:26:59.022015+0000","last_clean":"2023-07-14T19:26:59.022015+0000","last_became_active":"2023-07-14T19:26:56.498877+0000","last_became_peered":"2023-07-14T19:26:56.498877+0000","last_unstale":"2023-07-14T19:26:59.022015+0000","last_undegraded":"2023-07-14T19:26:59.022015+0000","last_fullsized":"2023-07-14T19:26:59.022015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:32:11.567666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307192+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:26:58.307192+0000","last_peered":"2023-07-14T19:26:58.307192+0000","last_clean":"2023-07-14T19:26:58.307192+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:26:58.307192+0000","last_undegraded":"2023-07-14T19:26:58.307192+0000","last_fullsized":"2023-07-14T19:26:58.307192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:22.256623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.777987+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:00.777987+0000","last_peered":"2023-07-14T19:27:00.777987+0000","last_clean":"2023-07-14T19:27:00.777987+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:00.777987+0000","last_undegraded":"2023-07-14T19:27:00.777987+0000","last_fullsized":"2023-07-14T19:27:00.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:40:59.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479968+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:26:58.479968+0000","last_peered":"2023-07-14T19:26:58.479968+0000","last_clean":"2023-07-14T19:26:58.479968+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:26:58.479968+0000","last_undegraded":"2023-07-14T19:26:58.479968+0000","last_fullsized":"2023-07-14T19:26:58.479968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:49:37.960423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.779679+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:01.779679+0000","last_peered":"2023-07-14T19:27:01.779679+0000","last_clean":"2023-07-14T19:27:01.779679+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:01.779679+0000","last_undegraded":"2023-07-14T19:27:01.779679+0000","last_fullsized":"2023-07-14T19:27:01.779679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:27:56.581809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479967+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:26:58.479967+0000","last_peered":"2023-07-14T19:26:58.479967+0000","last_clean":"2023-07-14T19:26:58.479967+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:26:58.479967+0000","last_undegraded":"2023-07-14T19:26:58.479967+0000","last_fullsized":"2023-07-14T19:26:58.479967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:34.136630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022492+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:26:59.022492+0000","last_peered":"2023-07-14T19:26:59.022492+0000","last_clean":"2023-07-14T19:26:59.022492+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:26:59.022492+0000","last_undegraded":"2023-07-14T19:26:59.022492+0000","last_fullsized":"2023-07-14T19:26:59.022492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:04.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479694+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:26:58.479694+0000","last_peered":"2023-07-14T19:26:58.479694+0000","last_clean":"2023-07-14T19:26:58.479694+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:26:58.479694+0000","last_undegraded":"2023-07-14T19:26:58.479694+0000","last_fullsized":"2023-07-14T19:26:58.479694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:06:27.371202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022069+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:26:59.022069+0000","last_peered":"2023-07-14T19:26:59.022069+0000","last_clean":"2023-07-14T19:26:59.022069+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:26:59.022069+0000","last_undegraded":"2023-07-14T19:26:59.022069+0000","last_fullsized":"2023-07-14T19:26:59.022069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:58.503189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.480077+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:26:58.480077+0000","last_peered":"2023-07-14T19:26:58.480077+0000","last_clean":"2023-07-14T19:26:58.480077+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:26:58.480077+0000","last_undegraded":"2023-07-14T19:26:58.480077+0000","last_fullsized":"2023-07-14T19:26:58.480077+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:11:43.314113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307087+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:26:58.307087+0000","last_peered":"2023-07-14T19:26:58.307087+0000","last_clean":"2023-07-14T19:26:58.307087+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:26:58.307087+0000","last_undegraded":"2023-07-14T19:26:58.307087+0000","last_fullsized":"2023-07-14T19:26:58.307087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:01:46.231283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021981+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:26:59.021981+0000","last_peered":"2023-07-14T19:26:59.021981+0000","last_clean":"2023-07-14T19:26:59.021981+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:26:59.021981+0000","last_undegraded":"2023-07-14T19:26:59.021981+0000","last_fullsized":"2023-07-14T19:26:59.021981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:23:56.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021869+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:26:59.021869+0000","last_peered":"2023-07-14T19:26:59.021869+0000","last_clean":"2023-07-14T19:26:59.021869+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:26:59.021869+0000","last_undegraded":"2023-07-14T19:26:59.021869+0000","last_fullsized":"2023-07-14T19:26:59.021869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:15.745903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022685+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:26:59.022685+0000","last_peered":"2023-07-14T19:26:59.022685+0000","last_clean":"2023-07-14T19:26:59.022685+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:26:59.022685+0000","last_undegraded":"2023-07-14T19:26:59.022685+0000","last_fullsized":"2023-07-14T19:26:59.022685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:02.290686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022283+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:26:59.022283+0000","last_peered":"2023-07-14T19:26:59.022283+0000","last_clean":"2023-07-14T19:26:59.022283+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:26:59.022283+0000","last_undegraded":"2023-07-14T19:26:59.022283+0000","last_fullsized":"2023-07-14T19:26:59.022283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:01:25.882894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307280+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:26:58.307280+0000","last_peered":"2023-07-14T19:26:58.307280+0000","last_clean":"2023-07-14T19:26:58.307280+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:26:58.307280+0000","last_undegraded":"2023-07-14T19:26:58.307280+0000","last_fullsized":"2023-07-14T19:26:58.307280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:30:29.404681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022701+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:26:59.022701+0000","last_peered":"2023-07-14T19:26:59.022701+0000","last_clean":"2023-07-14T19:26:59.022701+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:26:59.022701+0000","last_undegraded":"2023-07-14T19:26:59.022701+0000","last_fullsized":"2023-07-14T19:26:59.022701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:15.298866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307259+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:26:58.307259+0000","last_peered":"2023-07-14T19:26:58.307259+0000","last_clean":"2023-07-14T19:26:58.307259+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:26:58.307259+0000","last_undegraded":"2023-07-14T19:26:58.307259+0000","last_fullsized":"2023-07-14T19:26:58.307259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:07:49.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022919+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:26:59.022919+0000","last_peered":"2023-07-14T19:26:59.022919+0000","last_clean":"2023-07-14T19:26:59.022919+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:26:59.022919+0000","last_undegraded":"2023-07-14T19:26:59.022919+0000","last_fullsized":"2023-07-14T19:26:59.022919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:42:44.272349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307056+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:26:58.307056+0000","last_peered":"2023-07-14T19:26:58.307056+0000","last_clean":"2023-07-14T19:26:58.307056+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:26:58.307056+0000","last_undegraded":"2023-07-14T19:26:58.307056+0000","last_fullsized":"2023-07-14T19:26:58.307056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:26.245614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021790+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:26:59.021790+0000","last_peered":"2023-07-14T19:26:59.021790+0000","last_clean":"2023-07-14T19:26:59.021790+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:26:59.021790+0000","last_undegraded":"2023-07-14T19:26:59.021790+0000","last_fullsized":"2023-07-14T19:26:59.021790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:53:57.609137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022261+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:26:59.022261+0000","last_peered":"2023-07-14T19:26:59.022261+0000","last_clean":"2023-07-14T19:26:59.022261+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:26:59.022261+0000","last_undegraded":"2023-07-14T19:26:59.022261+0000","last_fullsized":"2023-07-14T19:26:59.022261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:29:15.878636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307091+0000","last_change":"2023-07-14T19:26:57.326341+0000","last_active":"2023-07-14T19:26:58.307091+0000","last_peered":"2023-07-14T19:26:58.307091+0000","last_clean":"2023-07-14T19:26:58.307091+0000","last_became_active":"2023-07-14T19:26:57.326042+0000","last_became_peered":"2023-07-14T19:26:57.326042+0000","last_unstale":"2023-07-14T19:26:58.307091+0000","last_undegraded":"2023-07-14T19:26:58.307091+0000","last_fullsized":"2023-07-14T19:26:58.307091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:06:39.460232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4504,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":101,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":14240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":414238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":937984,"data_stored":1006364,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629}]},{"osd":5,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":9,"up_from":64,"seq":274877906964,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":414820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.436}]},{"osd":4,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.69099999999999995}]},{"osd":8,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.622}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.755}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":414976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.442}]},{"osd":1,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.5}]},{"osd":3,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.53000000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83899999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.72499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":1005971,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.623}]},{"osd":2,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.82699999999999996}]},{"osd":3,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.61199999999999999,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.92500000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81999999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.60999999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.751,"15min":0.71099999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.8310000000000004,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.67500000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.88200000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":414238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41799999999999998,"15min":0.41899999999999998},"min":{"1min":0.30399999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.433},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.51100000000000001,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.628,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":1.427}]},{"osd":5,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68000000000000005,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.095,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.53500000000000003}]},{"osd":8,"last update":"Fri Jul 14 19:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.4019999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.4490000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":415802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.441,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Jul 14 19:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49399999999999999,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75,"5min":1.302,"15min":1.302},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.62}]},{"osd":4,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45900000000000002,"15min":0.44500000000000001},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.42699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.377,"15min":0.377},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85199999999999998,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.745},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90300000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.97699999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":414932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70199999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Jul 14 19:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.435,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Jul 14 19:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60899999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Jul 14 19:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":417104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.49099999999999999,"15min":0.46999999999999997},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.46500000000000002},"min":{"1min":0.42299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.72599999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.377,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.36399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":1.016,"15min":1.016},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.443,"15min":0.435},"min":{"1min":0.28699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81100000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":414522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.31900000000000001},"min":{"1min":0.29899999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.25900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.30299999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.40999999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.46100000000000002},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.32800000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.438,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41799999999999998}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920808,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":868352,"data_stored":414260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.28899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.48299999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.88900000000000001}]},{"osd":9,"last update":"Fri Jul 14 19:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.872},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":1004457,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.70199999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.996,"15min":0.996},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82099999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68999999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.443}]},{"osd":3,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":1.022,"15min":1.022},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":1.044,"15min":1.044},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.437,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88400000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86899999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":1.087,"15min":1.087},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.66600000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.871,"5min":1.113,"15min":1.113},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.876,"5min":1.004,"15min":1.004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67800000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.874},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.86199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:27:05.180 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T19:27:05.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/114393644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T19:27:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-14T19:27:05.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T19:27:05.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-07-14T19:27:05.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:04 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-07-14T19:27:05.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:05.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/114393644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:05.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-14T19:27:05.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-07-14T19:27:05.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-07-14T19:27:05.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2023-07-14T19:27:05.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:04 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-07-14T19:27:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 -- 172.21.15.82:0/1471555486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 msgr2=0x7efebc0ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 --2- 172.21.15.82:0/1471555486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0ff430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efeac009990 tx=0x7efeac009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 -- 172.21.15.82:0/1471555486 shutdown_connections 2023-07-14T19:27:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 --2- 172.21.15.82:0/1471555486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc1006e0 0x7efebc106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 --2- 172.21.15.82:0/1471555486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0ff430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 --2- 172.21.15.82:0/1471555486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc0ffd30 0x7efebc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 -- 172.21.15.82:0/1471555486 >> 172.21.15.82:0/1471555486 conn(0x7efebc0f98a0 msgr2=0x7efebc0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 -- 172.21.15.82:0/1471555486 shutdown_connections 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.328+0000 7efec1ac8700 1 -- 172.21.15.82:0/1471555486 wait complete. 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 Processor -- start 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 -- start start 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:05.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffd30 0x7efebc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc1006e0 0x7efebc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19d470 con 0x7efebc0ff020 2023-07-14T19:27:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19d5e0 con 0x7efebc0ffd30 2023-07-14T19:27:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efec1ac8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19d7a0 con 0x7efebc1006e0 2023-07-14T19:27:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efebb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efebb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0fe740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40932/0 (socket says 172.21.15.82:40932) 2023-07-14T19:27:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efebb7fe700 1 -- 172.21.15.82:0/3373365003 learned_addr learned my addr 172.21.15.82:0/3373365003 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efeb3fff700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffd30 0x7efebc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.329+0000 7efebbfff700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc1006e0 0x7efebc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:05.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efebb7fe700 1 -- 172.21.15.82:0/3373365003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc1006e0 msgr2=0x7efebc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efebb7fe700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc1006e0 0x7efebc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efebb7fe700 1 -- 172.21.15.82:0/3373365003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffd30 msgr2=0x7efebc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efebb7fe700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffd30 0x7efebc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efebb7fe700 1 -- 172.21.15.82:0/3373365003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0095f0 con 0x7efebc0ff020 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efeb3fff700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffd30 0x7efebc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efebb7fe700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0fe740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efeac004710 tx=0x7efeac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac01d070 con 0x7efebc0ff020 2023-07-14T19:27:05.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc19da20 con 0x7efebc0ff020 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeac021410 con 0x7efebc0ff020 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac017650 con 0x7efebc0ff020 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.330+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc19de50 con 0x7efebc0ff020 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.331+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc105a70 con 0x7efebc0ff020 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.332+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efeac021580 con 0x7efebc0ff020 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.332+0000 7efeb97fa700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efea80a12a0 0x7efea80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:05.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.333+0000 7efeb3fff700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efea80a12a0 0x7efea80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:05.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.333+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8124+0+0 (secure 0 0 0) 0x7efeac0cdd50 con 0x7efebc0ff020 2023-07-14T19:27:05.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.333+0000 7efeb3fff700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efea80a12a0 0x7efea80a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efebc0fed40 tx=0x7efea4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:05.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.337+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efeac097500 con 0x7efebc0ff020 2023-07-14T19:27:05.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.522+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efebc060a80 con 0x7efea80a12a0 2023-07-14T19:27:05.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.532+0000 7efeb97fa700 1 -- 172.21.15.82:0/3373365003 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335958 (secure 0 0 0) 0x7efebc060a80 con 0x7efea80a12a0 2023-07-14T19:27:05.536 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:05.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efea80a12a0 msgr2=0x7efea80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efea80a12a0 0x7efea80a3750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efebc0fed40 tx=0x7efea4009260 comp rx=0 tx=0).stop 2023-07-14T19:27:05.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 msgr2=0x7efebc0fe740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:05.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0fe740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efeac004710 tx=0x7efeac009580 comp rx=0 tx=0).stop 2023-07-14T19:27:05.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 shutdown_connections 2023-07-14T19:27:05.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efebc1006e0 0x7efebc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efea80a12a0 0x7efea80a3750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efebc0ff020 0x7efebc0fe740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 --2- 172.21.15.82:0/3373365003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efebc0ffd30 0x7efebc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:05.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 >> 172.21.15.82:0/3373365003 conn(0x7efebc0f98a0 msgr2=0x7efebc0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:05.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 shutdown_connections 2023-07-14T19:27:05.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:05.535+0000 7efec1ac8700 1 -- 172.21.15.82:0/3373365003 wait complete. 2023-07-14T19:27:05.540 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:27:05.548 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-07-14T19:27:04.778882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594872,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":141,"num_read_kb":124,"num_write":282,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334660,"kb_used_data":10420,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124607164,"statfs":{"total":1151940427776,"available":1151597735936,"internally_reserved":0,"allocated":10670080,"data_stored":6751684,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4504,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":101,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.474057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307225+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:26:58.307225+0000","last_peered":"2023-07-14T19:26:58.307225+0000","last_clean":"2023-07-14T19:26:58.307225+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:26:58.307225+0000","last_undegraded":"2023-07-14T19:26:58.307225+0000","last_fullsized":"2023-07-14T19:26:58.307225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:11:53.276829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022204+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:26:59.022204+0000","last_peered":"2023-07-14T19:26:59.022204+0000","last_clean":"2023-07-14T19:26:59.022204+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:26:59.022204+0000","last_undegraded":"2023-07-14T19:26:59.022204+0000","last_fullsized":"2023-07-14T19:26:59.022204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:52:37.455002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021492+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:26:59.021492+0000","last_peered":"2023-07-14T19:26:59.021492+0000","last_clean":"2023-07-14T19:26:59.021492+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:26:59.021492+0000","last_undegraded":"2023-07-14T19:26:59.021492+0000","last_fullsized":"2023-07-14T19:26:59.021492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:04:17.827893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022921+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:26:59.022921+0000","last_peered":"2023-07-14T19:26:59.022921+0000","last_clean":"2023-07-14T19:26:59.022921+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:26:59.022921+0000","last_undegraded":"2023-07-14T19:26:59.022921+0000","last_fullsized":"2023-07-14T19:26:59.022921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:57:42.948302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725619+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:26:58.725619+0000","last_peered":"2023-07-14T19:26:58.725619+0000","last_clean":"2023-07-14T19:26:58.725619+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:26:58.725619+0000","last_undegraded":"2023-07-14T19:26:58.725619+0000","last_fullsized":"2023-07-14T19:26:58.725619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:56.134158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308235+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:26:58.308235+0000","last_peered":"2023-07-14T19:26:58.308235+0000","last_clean":"2023-07-14T19:26:58.308235+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:26:58.308235+0000","last_undegraded":"2023-07-14T19:26:58.308235+0000","last_fullsized":"2023-07-14T19:26:58.308235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:12:55.795598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479553+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:26:58.479553+0000","last_peered":"2023-07-14T19:26:58.479553+0000","last_clean":"2023-07-14T19:26:58.479553+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:26:58.479553+0000","last_undegraded":"2023-07-14T19:26:58.479553+0000","last_fullsized":"2023-07-14T19:26:58.479553+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:03.769700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.306961+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:26:58.306961+0000","last_peered":"2023-07-14T19:26:58.306961+0000","last_clean":"2023-07-14T19:26:58.306961+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:26:58.306961+0000","last_undegraded":"2023-07-14T19:26:58.306961+0000","last_fullsized":"2023-07-14T19:26:58.306961+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:39:55.428976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022255+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:26:59.022255+0000","last_peered":"2023-07-14T19:26:59.022255+0000","last_clean":"2023-07-14T19:26:59.022255+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:26:59.022255+0000","last_undegraded":"2023-07-14T19:26:59.022255+0000","last_fullsized":"2023-07-14T19:26:59.022255+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:43:45.852784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308658+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:26:58.308658+0000","last_peered":"2023-07-14T19:26:58.308658+0000","last_clean":"2023-07-14T19:26:58.308658+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:26:58.308658+0000","last_undegraded":"2023-07-14T19:26:58.308658+0000","last_fullsized":"2023-07-14T19:26:58.308658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:54:35.306168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479735+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:26:58.479735+0000","last_peered":"2023-07-14T19:26:58.479735+0000","last_clean":"2023-07-14T19:26:58.479735+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:26:58.479735+0000","last_undegraded":"2023-07-14T19:26:58.479735+0000","last_fullsized":"2023-07-14T19:26:58.479735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:26:26.037572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022843+0000","last_change":"2023-07-14T19:26:57.324895+0000","last_active":"2023-07-14T19:26:59.022843+0000","last_peered":"2023-07-14T19:26:59.022843+0000","last_clean":"2023-07-14T19:26:59.022843+0000","last_became_active":"2023-07-14T19:26:57.324344+0000","last_became_peered":"2023-07-14T19:26:57.324344+0000","last_unstale":"2023-07-14T19:26:59.022843+0000","last_undegraded":"2023-07-14T19:26:59.022843+0000","last_fullsized":"2023-07-14T19:26:59.022843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:36:34.734566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022126+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:26:59.022126+0000","last_peered":"2023-07-14T19:26:59.022126+0000","last_clean":"2023-07-14T19:26:59.022126+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:26:59.022126+0000","last_undegraded":"2023-07-14T19:26:59.022126+0000","last_fullsized":"2023-07-14T19:26:59.022126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:17:01.906908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021558+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:26:59.021558+0000","last_peered":"2023-07-14T19:26:59.021558+0000","last_clean":"2023-07-14T19:26:59.021558+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:26:59.021558+0000","last_undegraded":"2023-07-14T19:26:59.021558+0000","last_fullsized":"2023-07-14T19:26:59.021558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:40:17.055224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021637+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:26:59.021637+0000","last_peered":"2023-07-14T19:26:59.021637+0000","last_clean":"2023-07-14T19:26:59.021637+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:26:59.021637+0000","last_undegraded":"2023-07-14T19:26:59.021637+0000","last_fullsized":"2023-07-14T19:26:59.021637+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:38:26.971168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021570+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:26:59.021570+0000","last_peered":"2023-07-14T19:26:59.021570+0000","last_clean":"2023-07-14T19:26:59.021570+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:26:59.021570+0000","last_undegraded":"2023-07-14T19:26:59.021570+0000","last_fullsized":"2023-07-14T19:26:59.021570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:35:38.758691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307380+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:26:58.307380+0000","last_peered":"2023-07-14T19:26:58.307380+0000","last_clean":"2023-07-14T19:26:58.307380+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:26:58.307380+0000","last_undegraded":"2023-07-14T19:26:58.307380+0000","last_fullsized":"2023-07-14T19:26:58.307380+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:21:48.872090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021520+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:26:59.021520+0000","last_peered":"2023-07-14T19:26:59.021520+0000","last_clean":"2023-07-14T19:26:59.021520+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:26:59.021520+0000","last_undegraded":"2023-07-14T19:26:59.021520+0000","last_fullsized":"2023-07-14T19:26:59.021520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:09:32.103654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022152+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:26:59.022152+0000","last_peered":"2023-07-14T19:26:59.022152+0000","last_clean":"2023-07-14T19:26:59.022152+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:26:59.022152+0000","last_undegraded":"2023-07-14T19:26:59.022152+0000","last_fullsized":"2023-07-14T19:26:59.022152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:54:36.812775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022120+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:26:59.022120+0000","last_peered":"2023-07-14T19:26:59.022120+0000","last_clean":"2023-07-14T19:26:59.022120+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:26:59.022120+0000","last_undegraded":"2023-07-14T19:26:59.022120+0000","last_fullsized":"2023-07-14T19:26:59.022120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:17.414571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307265+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:26:58.307265+0000","last_peered":"2023-07-14T19:26:58.307265+0000","last_clean":"2023-07-14T19:26:58.307265+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:26:58.307265+0000","last_undegraded":"2023-07-14T19:26:58.307265+0000","last_fullsized":"2023-07-14T19:26:58.307265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:31:46.472799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.717376+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:26:58.717376+0000","last_peered":"2023-07-14T19:26:58.717376+0000","last_clean":"2023-07-14T19:26:58.717376+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:26:58.717376+0000","last_undegraded":"2023-07-14T19:26:58.717376+0000","last_fullsized":"2023-07-14T19:26:58.717376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:56:17.440990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021987+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:26:59.021987+0000","last_peered":"2023-07-14T19:26:59.021987+0000","last_clean":"2023-07-14T19:26:59.021987+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:26:59.021987+0000","last_undegraded":"2023-07-14T19:26:59.021987+0000","last_fullsized":"2023-07-14T19:26:59.021987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:56:02.835550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.779342+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:01.779342+0000","last_peered":"2023-07-14T19:27:01.779342+0000","last_clean":"2023-07-14T19:27:01.779342+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:01.779342+0000","last_undegraded":"2023-07-14T19:27:01.779342+0000","last_fullsized":"2023-07-14T19:27:01.779342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:29:40.372439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021751+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:26:59.021751+0000","last_peered":"2023-07-14T19:26:59.021751+0000","last_clean":"2023-07-14T19:26:59.021751+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:26:59.021751+0000","last_undegraded":"2023-07-14T19:26:59.021751+0000","last_fullsized":"2023-07-14T19:26:59.021751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:02:32.436754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725830+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:26:58.725830+0000","last_peered":"2023-07-14T19:26:58.725830+0000","last_clean":"2023-07-14T19:26:58.725830+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:26:58.725830+0000","last_undegraded":"2023-07-14T19:26:58.725830+0000","last_fullsized":"2023-07-14T19:26:58.725830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:52:15.739288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022396+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:26:59.022396+0000","last_peered":"2023-07-14T19:26:59.022396+0000","last_clean":"2023-07-14T19:26:59.022396+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:26:59.022396+0000","last_undegraded":"2023-07-14T19:26:59.022396+0000","last_fullsized":"2023-07-14T19:26:59.022396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:56:29.503877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.777934+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:27:00.777934+0000","last_peered":"2023-07-14T19:27:00.777934+0000","last_clean":"2023-07-14T19:27:00.777934+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:27:00.777934+0000","last_undegraded":"2023-07-14T19:27:00.777934+0000","last_fullsized":"2023-07-14T19:27:00.777934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:15:04.045080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725644+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:26:58.725644+0000","last_peered":"2023-07-14T19:26:58.725644+0000","last_clean":"2023-07-14T19:26:58.725644+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:26:58.725644+0000","last_undegraded":"2023-07-14T19:26:58.725644+0000","last_fullsized":"2023-07-14T19:26:58.725644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:35:14.106188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308121+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:26:58.308121+0000","last_peered":"2023-07-14T19:26:58.308121+0000","last_clean":"2023-07-14T19:26:58.308121+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:26:58.308121+0000","last_undegraded":"2023-07-14T19:26:58.308121+0000","last_fullsized":"2023-07-14T19:26:58.308121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:57:10.165499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022669+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:26:59.022669+0000","last_peered":"2023-07-14T19:26:59.022669+0000","last_clean":"2023-07-14T19:26:59.022669+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:26:59.022669+0000","last_undegraded":"2023-07-14T19:26:59.022669+0000","last_fullsized":"2023-07-14T19:26:59.022669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:58:17.122017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.781505+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:01.781505+0000","last_peered":"2023-07-14T19:27:01.781505+0000","last_clean":"2023-07-14T19:27:01.781505+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:01.781505+0000","last_undegraded":"2023-07-14T19:27:01.781505+0000","last_fullsized":"2023-07-14T19:27:01.781505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:38:24.062279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307652+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:26:58.307652+0000","last_peered":"2023-07-14T19:26:58.307652+0000","last_clean":"2023-07-14T19:26:58.307652+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:26:58.307652+0000","last_undegraded":"2023-07-14T19:26:58.307652+0000","last_fullsized":"2023-07-14T19:26:58.307652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:37:33.497311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022394+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:26:59.022394+0000","last_peered":"2023-07-14T19:26:59.022394+0000","last_clean":"2023-07-14T19:26:59.022394+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:26:59.022394+0000","last_undegraded":"2023-07-14T19:26:59.022394+0000","last_fullsized":"2023-07-14T19:26:59.022394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:02.102706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021535+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:26:59.021535+0000","last_peered":"2023-07-14T19:26:59.021535+0000","last_clean":"2023-07-14T19:26:59.021535+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:26:59.021535+0000","last_undegraded":"2023-07-14T19:26:59.021535+0000","last_fullsized":"2023-07-14T19:26:59.021535+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:51:08.593239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.778707+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:27:00.778707+0000","last_peered":"2023-07-14T19:27:00.778707+0000","last_clean":"2023-07-14T19:27:00.778707+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:27:00.778707+0000","last_undegraded":"2023-07-14T19:27:00.778707+0000","last_fullsized":"2023-07-14T19:27:00.778707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:42:13.775871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022138+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:26:59.022138+0000","last_peered":"2023-07-14T19:26:59.022138+0000","last_clean":"2023-07-14T19:26:59.022138+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:26:59.022138+0000","last_undegraded":"2023-07-14T19:26:59.022138+0000","last_fullsized":"2023-07-14T19:26:59.022138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:15:26.311344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.777334+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:00.777334+0000","last_peered":"2023-07-14T19:27:00.777334+0000","last_clean":"2023-07-14T19:27:00.777334+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:00.777334+0000","last_undegraded":"2023-07-14T19:27:00.777334+0000","last_fullsized":"2023-07-14T19:27:00.777334+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:23:41.881340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021853+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:26:59.021853+0000","last_peered":"2023-07-14T19:26:59.021853+0000","last_clean":"2023-07-14T19:26:59.021853+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:26:59.021853+0000","last_undegraded":"2023-07-14T19:26:59.021853+0000","last_fullsized":"2023-07-14T19:26:59.021853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:11:34.073631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022610+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:26:59.022610+0000","last_peered":"2023-07-14T19:26:59.022610+0000","last_clean":"2023-07-14T19:26:59.022610+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:26:59.022610+0000","last_undegraded":"2023-07-14T19:26:59.022610+0000","last_fullsized":"2023-07-14T19:26:59.022610+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:47:15.187668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021910+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:26:59.021910+0000","last_peered":"2023-07-14T19:26:59.021910+0000","last_clean":"2023-07-14T19:26:59.021910+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:26:59.021910+0000","last_undegraded":"2023-07-14T19:26:59.021910+0000","last_fullsized":"2023-07-14T19:26:59.021910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:12:50.801527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'8","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.779373+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:27:00.779373+0000","last_peered":"2023-07-14T19:27:00.779373+0000","last_clean":"2023-07-14T19:27:00.779373+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:27:00.779373+0000","last_undegraded":"2023-07-14T19:27:00.779373+0000","last_fullsized":"2023-07-14T19:27:00.779373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:32:12.785323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725819+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:26:58.725819+0000","last_peered":"2023-07-14T19:26:58.725819+0000","last_clean":"2023-07-14T19:26:58.725819+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:26:58.725819+0000","last_undegraded":"2023-07-14T19:26:58.725819+0000","last_fullsized":"2023-07-14T19:26:58.725819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:38.092509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.780074+0000","last_change":"2023-07-14T19:26:56.500601+0000","last_active":"2023-07-14T19:27:01.780074+0000","last_peered":"2023-07-14T19:27:01.780074+0000","last_clean":"2023-07-14T19:27:01.780074+0000","last_became_active":"2023-07-14T19:26:56.500296+0000","last_became_peered":"2023-07-14T19:26:56.500296+0000","last_unstale":"2023-07-14T19:27:01.780074+0000","last_undegraded":"2023-07-14T19:27:01.780074+0000","last_fullsized":"2023-07-14T19:27:01.780074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:54:33.136846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022264+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:26:59.022264+0000","last_peered":"2023-07-14T19:26:59.022264+0000","last_clean":"2023-07-14T19:26:59.022264+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:26:59.022264+0000","last_undegraded":"2023-07-14T19:26:59.022264+0000","last_fullsized":"2023-07-14T19:26:59.022264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:02:31.957284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.776790+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:00.776790+0000","last_peered":"2023-07-14T19:27:00.776790+0000","last_clean":"2023-07-14T19:27:00.776790+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:00.776790+0000","last_undegraded":"2023-07-14T19:27:00.776790+0000","last_fullsized":"2023-07-14T19:27:00.776790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:34.301368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479787+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:26:58.479787+0000","last_peered":"2023-07-14T19:26:58.479787+0000","last_clean":"2023-07-14T19:26:58.479787+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:26:58.479787+0000","last_undegraded":"2023-07-14T19:26:58.479787+0000","last_fullsized":"2023-07-14T19:26:58.479787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:05:40.828316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022639+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:26:59.022639+0000","last_peered":"2023-07-14T19:26:59.022639+0000","last_clean":"2023-07-14T19:26:59.022639+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:26:59.022639+0000","last_undegraded":"2023-07-14T19:26:59.022639+0000","last_fullsized":"2023-07-14T19:26:59.022639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:25.824522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479466+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:26:58.479466+0000","last_peered":"2023-07-14T19:26:58.479466+0000","last_clean":"2023-07-14T19:26:58.479466+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:26:58.479466+0000","last_undegraded":"2023-07-14T19:26:58.479466+0000","last_fullsized":"2023-07-14T19:26:58.479466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:10:44.012832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.781194+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:01.781194+0000","last_peered":"2023-07-14T19:27:01.781194+0000","last_clean":"2023-07-14T19:27:01.781194+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:01.781194+0000","last_undegraded":"2023-07-14T19:27:01.781194+0000","last_fullsized":"2023-07-14T19:27:01.781194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:07:24.913669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022500+0000","last_change":"2023-07-14T19:26:57.323495+0000","last_active":"2023-07-14T19:26:59.022500+0000","last_peered":"2023-07-14T19:26:59.022500+0000","last_clean":"2023-07-14T19:26:59.022500+0000","last_became_active":"2023-07-14T19:26:57.323199+0000","last_became_peered":"2023-07-14T19:26:57.323199+0000","last_unstale":"2023-07-14T19:26:59.022500+0000","last_undegraded":"2023-07-14T19:26:59.022500+0000","last_fullsized":"2023-07-14T19:26:59.022500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:11:39.678080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725961+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:26:58.725961+0000","last_peered":"2023-07-14T19:26:58.725961+0000","last_clean":"2023-07-14T19:26:58.725961+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:26:58.725961+0000","last_undegraded":"2023-07-14T19:26:58.725961+0000","last_fullsized":"2023-07-14T19:26:58.725961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:32:35.481135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307225+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:26:58.307225+0000","last_peered":"2023-07-14T19:26:58.307225+0000","last_clean":"2023-07-14T19:26:58.307225+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:26:58.307225+0000","last_undegraded":"2023-07-14T19:26:58.307225+0000","last_fullsized":"2023-07-14T19:26:58.307225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:53:59.627293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021920+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:26:59.021920+0000","last_peered":"2023-07-14T19:26:59.021920+0000","last_clean":"2023-07-14T19:26:59.021920+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:26:59.021920+0000","last_undegraded":"2023-07-14T19:26:59.021920+0000","last_fullsized":"2023-07-14T19:26:59.021920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:48.402550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308578+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:26:58.308578+0000","last_peered":"2023-07-14T19:26:58.308578+0000","last_clean":"2023-07-14T19:26:58.308578+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:26:58.308578+0000","last_undegraded":"2023-07-14T19:26:58.308578+0000","last_fullsized":"2023-07-14T19:26:58.308578+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:59:10.880240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479879+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:26:58.479879+0000","last_peered":"2023-07-14T19:26:58.479879+0000","last_clean":"2023-07-14T19:26:58.479879+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:26:58.479879+0000","last_undegraded":"2023-07-14T19:26:58.479879+0000","last_fullsized":"2023-07-14T19:26:58.479879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:33:39.197230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.308501+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:26:58.308501+0000","last_peered":"2023-07-14T19:26:58.308501+0000","last_clean":"2023-07-14T19:26:58.308501+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:26:58.308501+0000","last_undegraded":"2023-07-14T19:26:58.308501+0000","last_fullsized":"2023-07-14T19:26:58.308501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:20:26.973280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022680+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:26:59.022680+0000","last_peered":"2023-07-14T19:26:59.022680+0000","last_clean":"2023-07-14T19:26:59.022680+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:26:59.022680+0000","last_undegraded":"2023-07-14T19:26:59.022680+0000","last_fullsized":"2023-07-14T19:26:59.022680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:31:01.985023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.780434+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:27:01.780434+0000","last_peered":"2023-07-14T19:27:01.780434+0000","last_clean":"2023-07-14T19:27:01.780434+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:27:01.780434+0000","last_undegraded":"2023-07-14T19:27:01.780434+0000","last_fullsized":"2023-07-14T19:27:01.780434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:42:58.103911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307562+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:26:58.307562+0000","last_peered":"2023-07-14T19:26:58.307562+0000","last_clean":"2023-07-14T19:26:58.307562+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:26:58.307562+0000","last_undegraded":"2023-07-14T19:26:58.307562+0000","last_fullsized":"2023-07-14T19:26:58.307562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:02:19.231493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022845+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:26:59.022845+0000","last_peered":"2023-07-14T19:26:59.022845+0000","last_clean":"2023-07-14T19:26:59.022845+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:26:59.022845+0000","last_undegraded":"2023-07-14T19:26:59.022845+0000","last_fullsized":"2023-07-14T19:26:59.022845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:05:27.875468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":339,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479508+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:26:58.479508+0000","last_peered":"2023-07-14T19:26:58.479508+0000","last_clean":"2023-07-14T19:26:58.479508+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:26:58.479508+0000","last_undegraded":"2023-07-14T19:26:58.479508+0000","last_fullsized":"2023-07-14T19:26:58.479508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:56.740236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021287+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:26:59.021287+0000","last_peered":"2023-07-14T19:26:59.021287+0000","last_clean":"2023-07-14T19:26:59.021287+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:26:59.021287+0000","last_undegraded":"2023-07-14T19:26:59.021287+0000","last_fullsized":"2023-07-14T19:26:59.021287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:41:18.800844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.779712+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:01.779712+0000","last_peered":"2023-07-14T19:27:01.779712+0000","last_clean":"2023-07-14T19:27:01.779712+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:01.779712+0000","last_undegraded":"2023-07-14T19:27:01.779712+0000","last_fullsized":"2023-07-14T19:27:01.779712+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:03:59.089182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.776988+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:00.776988+0000","last_peered":"2023-07-14T19:27:00.776988+0000","last_clean":"2023-07-14T19:27:00.776988+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:00.776988+0000","last_undegraded":"2023-07-14T19:27:00.776988+0000","last_fullsized":"2023-07-14T19:27:00.776988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:58:01.879837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022144+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:26:59.022144+0000","last_peered":"2023-07-14T19:26:59.022144+0000","last_clean":"2023-07-14T19:26:59.022144+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:26:59.022144+0000","last_undegraded":"2023-07-14T19:26:59.022144+0000","last_fullsized":"2023-07-14T19:26:59.022144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:25:33.109783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022034+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:26:59.022034+0000","last_peered":"2023-07-14T19:26:59.022034+0000","last_clean":"2023-07-14T19:26:59.022034+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:26:59.022034+0000","last_undegraded":"2023-07-14T19:26:59.022034+0000","last_fullsized":"2023-07-14T19:26:59.022034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:46:17.438048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307289+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:26:58.307289+0000","last_peered":"2023-07-14T19:26:58.307289+0000","last_clean":"2023-07-14T19:26:58.307289+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:26:58.307289+0000","last_undegraded":"2023-07-14T19:26:58.307289+0000","last_fullsized":"2023-07-14T19:26:58.307289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:16:17.016308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.023007+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:26:59.023007+0000","last_peered":"2023-07-14T19:26:59.023007+0000","last_clean":"2023-07-14T19:26:59.023007+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:26:59.023007+0000","last_undegraded":"2023-07-14T19:26:59.023007+0000","last_fullsized":"2023-07-14T19:26:59.023007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:39:04.731889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022327+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:26:59.022327+0000","last_peered":"2023-07-14T19:26:59.022327+0000","last_clean":"2023-07-14T19:26:59.022327+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:26:59.022327+0000","last_undegraded":"2023-07-14T19:26:59.022327+0000","last_fullsized":"2023-07-14T19:26:59.022327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:45.416568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.775491+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:00.775491+0000","last_peered":"2023-07-14T19:27:00.775491+0000","last_clean":"2023-07-14T19:27:00.775491+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:00.775491+0000","last_undegraded":"2023-07-14T19:27:00.775491+0000","last_fullsized":"2023-07-14T19:27:00.775491+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:13:29.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.725512+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:26:58.725512+0000","last_peered":"2023-07-14T19:26:58.725512+0000","last_clean":"2023-07-14T19:26:58.725512+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:26:58.725512+0000","last_undegraded":"2023-07-14T19:26:58.725512+0000","last_fullsized":"2023-07-14T19:26:58.725512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:19:35.106302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.776125+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:00.776125+0000","last_peered":"2023-07-14T19:27:00.776125+0000","last_clean":"2023-07-14T19:27:00.776125+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:00.776125+0000","last_undegraded":"2023-07-14T19:27:00.776125+0000","last_fullsized":"2023-07-14T19:27:00.776125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:20:19.370162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307014+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:26:58.307014+0000","last_peered":"2023-07-14T19:26:58.307014+0000","last_clean":"2023-07-14T19:26:58.307014+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:26:58.307014+0000","last_undegraded":"2023-07-14T19:26:58.307014+0000","last_fullsized":"2023-07-14T19:26:58.307014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:42:53.548772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022015+0000","last_change":"2023-07-14T19:26:56.499194+0000","last_active":"2023-07-14T19:26:59.022015+0000","last_peered":"2023-07-14T19:26:59.022015+0000","last_clean":"2023-07-14T19:26:59.022015+0000","last_became_active":"2023-07-14T19:26:56.498877+0000","last_became_peered":"2023-07-14T19:26:56.498877+0000","last_unstale":"2023-07-14T19:26:59.022015+0000","last_undegraded":"2023-07-14T19:26:59.022015+0000","last_fullsized":"2023-07-14T19:26:59.022015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:32:11.567666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307192+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:26:58.307192+0000","last_peered":"2023-07-14T19:26:58.307192+0000","last_clean":"2023-07-14T19:26:58.307192+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:26:58.307192+0000","last_undegraded":"2023-07-14T19:26:58.307192+0000","last_fullsized":"2023-07-14T19:26:58.307192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:59:22.256623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:00.777987+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:00.777987+0000","last_peered":"2023-07-14T19:27:00.777987+0000","last_clean":"2023-07-14T19:27:00.777987+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:00.777987+0000","last_undegraded":"2023-07-14T19:27:00.777987+0000","last_fullsized":"2023-07-14T19:27:00.777987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:40:59.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479968+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:26:58.479968+0000","last_peered":"2023-07-14T19:26:58.479968+0000","last_clean":"2023-07-14T19:26:58.479968+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:26:58.479968+0000","last_undegraded":"2023-07-14T19:26:58.479968+0000","last_fullsized":"2023-07-14T19:26:58.479968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:49:37.960423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:27:01.779679+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:01.779679+0000","last_peered":"2023-07-14T19:27:01.779679+0000","last_clean":"2023-07-14T19:27:01.779679+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:01.779679+0000","last_undegraded":"2023-07-14T19:27:01.779679+0000","last_fullsized":"2023-07-14T19:27:01.779679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:27:56.581809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479967+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:26:58.479967+0000","last_peered":"2023-07-14T19:26:58.479967+0000","last_clean":"2023-07-14T19:26:58.479967+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:26:58.479967+0000","last_undegraded":"2023-07-14T19:26:58.479967+0000","last_fullsized":"2023-07-14T19:26:58.479967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:49:34.136630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022492+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:26:59.022492+0000","last_peered":"2023-07-14T19:26:59.022492+0000","last_clean":"2023-07-14T19:26:59.022492+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:26:59.022492+0000","last_undegraded":"2023-07-14T19:26:59.022492+0000","last_fullsized":"2023-07-14T19:26:59.022492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:20:04.746296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.479694+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:26:58.479694+0000","last_peered":"2023-07-14T19:26:58.479694+0000","last_clean":"2023-07-14T19:26:58.479694+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:26:58.479694+0000","last_undegraded":"2023-07-14T19:26:58.479694+0000","last_fullsized":"2023-07-14T19:26:58.479694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:06:27.371202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022069+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:26:59.022069+0000","last_peered":"2023-07-14T19:26:59.022069+0000","last_clean":"2023-07-14T19:26:59.022069+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:26:59.022069+0000","last_undegraded":"2023-07-14T19:26:59.022069+0000","last_fullsized":"2023-07-14T19:26:59.022069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:04:58.503189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.480077+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:26:58.480077+0000","last_peered":"2023-07-14T19:26:58.480077+0000","last_clean":"2023-07-14T19:26:58.480077+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:26:58.480077+0000","last_undegraded":"2023-07-14T19:26:58.480077+0000","last_fullsized":"2023-07-14T19:26:58.480077+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:11:43.314113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307087+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:26:58.307087+0000","last_peered":"2023-07-14T19:26:58.307087+0000","last_clean":"2023-07-14T19:26:58.307087+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:26:58.307087+0000","last_undegraded":"2023-07-14T19:26:58.307087+0000","last_fullsized":"2023-07-14T19:26:58.307087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:01:46.231283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021981+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:26:59.021981+0000","last_peered":"2023-07-14T19:26:59.021981+0000","last_clean":"2023-07-14T19:26:59.021981+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:26:59.021981+0000","last_undegraded":"2023-07-14T19:26:59.021981+0000","last_fullsized":"2023-07-14T19:26:59.021981+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:23:56.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021869+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:26:59.021869+0000","last_peered":"2023-07-14T19:26:59.021869+0000","last_clean":"2023-07-14T19:26:59.021869+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:26:59.021869+0000","last_undegraded":"2023-07-14T19:26:59.021869+0000","last_fullsized":"2023-07-14T19:26:59.021869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:15:15.745903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022685+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:26:59.022685+0000","last_peered":"2023-07-14T19:26:59.022685+0000","last_clean":"2023-07-14T19:26:59.022685+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:26:59.022685+0000","last_undegraded":"2023-07-14T19:26:59.022685+0000","last_fullsized":"2023-07-14T19:26:59.022685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:47:02.290686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022283+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:26:59.022283+0000","last_peered":"2023-07-14T19:26:59.022283+0000","last_clean":"2023-07-14T19:26:59.022283+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:26:59.022283+0000","last_undegraded":"2023-07-14T19:26:59.022283+0000","last_fullsized":"2023-07-14T19:26:59.022283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:01:25.882894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307280+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:26:58.307280+0000","last_peered":"2023-07-14T19:26:58.307280+0000","last_clean":"2023-07-14T19:26:58.307280+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:26:58.307280+0000","last_undegraded":"2023-07-14T19:26:58.307280+0000","last_fullsized":"2023-07-14T19:26:58.307280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:30:29.404681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022701+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:26:59.022701+0000","last_peered":"2023-07-14T19:26:59.022701+0000","last_clean":"2023-07-14T19:26:59.022701+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:26:59.022701+0000","last_undegraded":"2023-07-14T19:26:59.022701+0000","last_fullsized":"2023-07-14T19:26:59.022701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:09:15.298866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307259+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:26:58.307259+0000","last_peered":"2023-07-14T19:26:58.307259+0000","last_clean":"2023-07-14T19:26:58.307259+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:26:58.307259+0000","last_undegraded":"2023-07-14T19:26:58.307259+0000","last_fullsized":"2023-07-14T19:26:58.307259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:07:49.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022919+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:26:59.022919+0000","last_peered":"2023-07-14T19:26:59.022919+0000","last_clean":"2023-07-14T19:26:59.022919+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:26:59.022919+0000","last_undegraded":"2023-07-14T19:26:59.022919+0000","last_fullsized":"2023-07-14T19:26:59.022919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:42:44.272349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307056+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:26:58.307056+0000","last_peered":"2023-07-14T19:26:58.307056+0000","last_clean":"2023-07-14T19:26:58.307056+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:26:58.307056+0000","last_undegraded":"2023-07-14T19:26:58.307056+0000","last_fullsized":"2023-07-14T19:26:58.307056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:10:26.245614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.021790+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:26:59.021790+0000","last_peered":"2023-07-14T19:26:59.021790+0000","last_clean":"2023-07-14T19:26:59.021790+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:26:59.021790+0000","last_undegraded":"2023-07-14T19:26:59.021790+0000","last_fullsized":"2023-07-14T19:26:59.021790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:53:57.609137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:59.022261+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:26:59.022261+0000","last_peered":"2023-07-14T19:26:59.022261+0000","last_clean":"2023-07-14T19:26:59.022261+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:26:59.022261+0000","last_undegraded":"2023-07-14T19:26:59.022261+0000","last_fullsized":"2023-07-14T19:26:59.022261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:29:15.878636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-07-14T19:26:58.307091+0000","last_change":"2023-07-14T19:26:57.326341+0000","last_active":"2023-07-14T19:26:58.307091+0000","last_peered":"2023-07-14T19:26:58.307091+0000","last_clean":"2023-07-14T19:26:58.307091+0000","last_became_active":"2023-07-14T19:26:57.326042+0000","last_became_peered":"2023-07-14T19:26:57.326042+0000","last_unstale":"2023-07-14T19:26:58.307091+0000","last_undegraded":"2023-07-14T19:26:58.307091+0000","last_fullsized":"2023-07-14T19:26:58.307091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:06:39.460232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4504,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":101,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":14240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514508,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":414238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":10,"up_from":69,"seq":296352743440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":937984,"data_stored":1006364,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629}]},{"osd":5,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":9,"up_from":64,"seq":274877906964,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":868352,"data_stored":414820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.77900000000000003}]},{"osd":2,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.436}]},{"osd":4,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.69099999999999995}]},{"osd":8,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.622}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.755}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":414976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.442}]},{"osd":1,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.5}]},{"osd":3,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.53000000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83899999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.72499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]}]},{"osd":7,"up_from":50,"seq":214748364829,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":1005971,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.623}]},{"osd":2,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.82699999999999996}]},{"osd":3,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.61199999999999999,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.92500000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81999999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.60999999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.751,"15min":0.71099999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.8310000000000004,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.67500000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.88200000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.96699999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":864256,"data_stored":414238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41799999999999998,"15min":0.41899999999999998},"min":{"1min":0.30399999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.27000000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56100000000000005,"15min":0.54700000000000004},"min":{"1min":0.38800000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.433},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.51100000000000001,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.628,"15min":0.622},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":1.427}]},{"osd":5,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68000000000000005,"15min":0.67700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.095,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.53500000000000003}]},{"osd":8,"last update":"Fri Jul 14 19:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.4019999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.4490000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":415802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.441,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Jul 14 19:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.49399999999999999,"15min":0.47899999999999998},"min":{"1min":0.38200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.315,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75,"5min":1.302,"15min":1.302},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.85699999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.62}]},{"osd":4,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45900000000000002,"15min":0.44500000000000001},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57399999999999995,"15min":0.56599999999999995},"min":{"1min":0.42699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.377,"15min":0.377},"max":{"1min":0.755,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58299999999999996,"15min":0.58399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.441,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85199999999999998,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.745},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.90300000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.97699999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.95099999999999996}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":876544,"data_stored":414932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70199999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Jul 14 19:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.435,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.74099999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Jul 14 19:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60899999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Jul 14 19:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:26:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":417104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.49099999999999999,"15min":0.46999999999999997},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.46500000000000002},"min":{"1min":0.42299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.72599999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.497},"min":{"1min":0.377,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51300000000000001,"15min":0.499},"min":{"1min":0.36399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57799999999999996,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77300000000000002,"5min":1.016,"15min":1.016},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.443,"15min":0.435},"min":{"1min":0.28699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Jul 14 19:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81100000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.504,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.76300000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.70899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":414522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.33000000000000002,"15min":0.31900000000000001},"min":{"1min":0.29899999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.33400000000000002},"min":{"1min":0.25900000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.30299999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.38,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.40999999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.46100000000000002},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.32800000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.438,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.41799999999999998}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920808,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":868352,"data_stored":414260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.28899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.48299999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:26:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.47499999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64400000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.88900000000000001}]},{"osd":9,"last update":"Fri Jul 14 19:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.93100000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.872},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757285,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":1004457,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.70199999999999996,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.996,"15min":0.996},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.38300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82099999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42399999999999999,"15min":0.41999999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68999999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29999999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63800000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.443}]},{"osd":3,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":1.022,"15min":1.022},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78900000000000003,"5min":1.044,"15min":1.044},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.437,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88400000000000001,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.754},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.40400000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86899999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":1.087,"15min":1.087},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66900000000000004,"15min":0.66600000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.871,"5min":1.113,"15min":1.113},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.876,"5min":1.004,"15min":1.004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:26:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67800000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.874},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.86199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:27:05.561 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-14T19:27:05.562 INFO:tasks.cephadm:Setting up client nodes... 2023-07-14T19:27:05.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-14T19:27:05.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: pgmap v251: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 3.7 KiB/s wr, 13 op/s 2023-07-14T19:27:05.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T19:27:05.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-14T19:27:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T19:27:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-07-14T19:27:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-07-14T19:27:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T19:27:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-14T19:27:05.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2808613849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:05.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:05 smithi082 ceph-mon[127461]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:06.137 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: pgmap v251: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 3.7 KiB/s wr, 13 op/s 2023-07-14T19:27:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T19:27:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-14T19:27:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T19:27:06.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-07-14T19:27:06.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-07-14T19:27:06.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T19:27:06.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-14T19:27:06.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2808613849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:06.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:05 smithi133 ceph-mon[128894]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: pgmap v251: 97 pgs: 97 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 3.7 KiB/s wr, 13 op/s 2023-07-14T19:27:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-14T19:27:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-07-14T19:27:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-07-14T19:27:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2023-07-14T19:27:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-07-14T19:27:06.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: osdmap e82: 12 total, 12 up, 12 in 2023-07-14T19:27:06.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-14T19:27:06.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2808613849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:06.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:05 smithi161 ceph-mon[129184]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:06.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:06 smithi082 ceph-mon[127461]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:06.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:06 smithi082 ceph-mon[127461]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T19:27:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:06 smithi133 ceph-mon[128894]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:07.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:06 smithi133 ceph-mon[128894]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T19:27:07.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:06 smithi161 ceph-mon[129184]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:07.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:06 smithi161 ceph-mon[129184]: osdmap e83: 12 total, 12 up, 12 in 2023-07-14T19:27:07.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.427+0000 7f97f21a3700 1 -- 172.21.15.82:0/955965186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0febe0 msgr2=0x7f97ec0feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.427+0000 7f97f21a3700 1 --2- 172.21.15.82:0/955965186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0febe0 0x7f97ec0feff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009990 tx=0x7f97dc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 -- 172.21.15.82:0/955965186 shutdown_connections 2023-07-14T19:27:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 --2- 172.21.15.82:0/955965186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec100770 0x7f97ec107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 --2- 172.21.15.82:0/955965186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0febe0 0x7f97ec0feff0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 --2- 172.21.15.82:0/955965186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec0ffde0 0x7f97ec100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 -- 172.21.15.82:0/955965186 >> 172.21.15.82:0/955965186 conn(0x7f97ec0fa210 msgr2=0x7f97ec0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:07.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 -- 172.21.15.82:0/955965186 shutdown_connections 2023-07-14T19:27:07.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 -- 172.21.15.82:0/955965186 wait complete. 2023-07-14T19:27:07.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 Processor -- start 2023-07-14T19:27:07.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.428+0000 7f97f21a3700 1 -- start start 2023-07-14T19:27:07.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec0febe0 0x7f97ec194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 0x7f97ec194ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec100770 0x7f97ec19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19c4f0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19c630 con 0x7f97ec100770 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec19c7c0 con 0x7f97ec0febe0 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 0x7f97ec194ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:07.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 0x7f97ec194ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40946/0 (socket says 172.21.15.82:40946) 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 -- 172.21.15.82:0/2379302160 learned_addr learned my addr 172.21.15.82:0/2379302160 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eb7fe700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec0febe0 0x7f97ec194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97ebfff700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec100770 0x7f97ec19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 -- 172.21.15.82:0/2379302160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec0febe0 msgr2=0x7f97ec194960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec0febe0 0x7f97ec194960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 -- 172.21.15.82:0/2379302160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec100770 msgr2=0x7f97ec19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec100770 0x7f97ec19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 -- 172.21.15.82:0/2379302160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc0095f0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97eaffd700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 0x7f97ec194ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c950 tx=0x7f97e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00078d0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec068160 con 0x7f97ec0ffde0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e000f450 con 0x7f97ec0ffde0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0018660 con 0x7f97ec0ffde0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.429+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec0686b0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.430+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec10a3a0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.431+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f97e0007a30 con 0x7f97ec0ffde0 2023-07-14T19:27:07.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.432+0000 7f97e8ff9700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97d40a11e0 0x7f97d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:07.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.432+0000 7f97eb7fe700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97d40a11e0 0x7f97d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:07.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.432+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f97e00cc7b0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.432+0000 7f97eb7fe700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97d40a11e0 0x7f97d40a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00b2e0 tx=0x7f97dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:07.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.436+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f97e0095ec0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.703+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f97ec106030 con 0x7f97ec0ffde0 2023-07-14T19:27:07.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.707+0000 7f97e8ff9700 1 -- 172.21.15.82:0/2379302160 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f97e00994e0 con 0x7f97ec0ffde0 2023-07-14T19:27:07.709 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2023-07-14T19:27:07.709 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQCLobFkyJz8KRAACv/pSkACFLXk7Mm5UV76Cw== 2023-07-14T19:27:07.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97d40a11e0 msgr2=0x7f97d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:07.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97d40a11e0 0x7f97d40a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00b2e0 tx=0x7f97dc005fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 msgr2=0x7f97ec194ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:07.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 0x7f97ec194ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c950 tx=0x7f97e000cc60 comp rx=0 tx=0).stop 2023-07-14T19:27:07.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 shutdown_connections 2023-07-14T19:27:07.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97ec100770 0x7f97ec19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97ec0febe0 0x7f97ec194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97d40a11e0 0x7f97d40a3690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 --2- 172.21.15.82:0/2379302160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97ec0ffde0 0x7f97ec194ea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:07.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 >> 172.21.15.82:0/2379302160 conn(0x7f97ec0fa210 msgr2=0x7f97ec103370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:07.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 shutdown_connections 2023-07-14T19:27:07.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:07.710+0000 7f97f21a3700 1 -- 172.21.15.82:0/2379302160 wait complete. 2023-07-14T19:27:08.178 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:07 smithi082 ceph-mon[127461]: pgmap v254: 97 pgs: 2 peering, 95 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.2 KiB/s wr, 19 op/s 2023-07-14T19:27:08.178 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:07 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2379302160' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T19:27:08.178 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:07 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2379302160' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T19:27:08.180 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:27:08.181 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-14T19:27:08.181 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-14T19:27:08.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:07 smithi133 ceph-mon[128894]: pgmap v254: 97 pgs: 2 peering, 95 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.2 KiB/s wr, 19 op/s 2023-07-14T19:27:08.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:07 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2379302160' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T19:27:08.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:07 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2379302160' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T19:27:08.221 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-14T19:27:08.222 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-14T19:27:08.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph mgr dump --format=json 2023-07-14T19:27:08.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:07 smithi161 ceph-mon[129184]: pgmap v254: 97 pgs: 2 peering, 95 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.2 KiB/s wr, 19 op/s 2023-07-14T19:27:08.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:07 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2379302160' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-14T19:27:08.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:07 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2379302160' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-14T19:27:08.811 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:10.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.028+0000 7faeccda2700 1 -- 172.21.15.82:0/3616964786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 msgr2=0x7faec8100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:10.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.028+0000 7faeccda2700 1 --2- 172.21.15.82:0/3616964786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8100300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faeb80099c0 tx=0x7faeb8009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 -- 172.21.15.82:0/3616964786 shutdown_connections 2023-07-14T19:27:10.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 --2- 172.21.15.82:0/3616964786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec8104fd0 0x7faec810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 --2- 172.21.15.82:0/3616964786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8100300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 --2- 172.21.15.82:0/3616964786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec81008d0 0x7faec81048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.032 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 -- 172.21.15.82:0/3616964786 >> 172.21.15.82:0/3616964786 conn(0x7faec80fb790 msgr2=0x7faec80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 -- 172.21.15.82:0/3616964786 shutdown_connections 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 -- 172.21.15.82:0/3616964786 wait complete. 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 Processor -- start 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.029+0000 7faeccda2700 1 -- start start 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec81008d0 0x7faec8196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:10.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec8104fd0 0x7faec819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec819adc0 con 0x7faec80fff30 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec819af30 con 0x7faec8104fd0 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec819b0f0 con 0x7faec81008d0 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40956/0 (socket says 172.21.15.82:40956) 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 -- 172.21.15.82:0/342608628 learned_addr learned my addr 172.21.15.82:0/342608628 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:10.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec6d9d700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec8104fd0 0x7faec819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec5d9b700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec81008d0 0x7faec8196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 -- 172.21.15.82:0/342608628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec81008d0 msgr2=0x7faec8196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec81008d0 0x7faec8196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 -- 172.21.15.82:0/342608628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec8104fd0 msgr2=0x7faec819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec8104fd0 0x7faec819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 -- 172.21.15.82:0/342608628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80095f0 con 0x7faec80fff30 2023-07-14T19:27:10.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec5d9b700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec81008d0 0x7faec8196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:10.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faec659c700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8195e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009990 tx=0x7faeb8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:10.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb801d070 con 0x7faec80fff30 2023-07-14T19:27:10.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faeb8003f90 con 0x7faec80fff30 2023-07-14T19:27:10.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec819b370 con 0x7faec80fff30 2023-07-14T19:27:10.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8022ce0 con 0x7faec80fff30 2023-07-14T19:27:10.036 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.030+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec81a2c30 con 0x7faec80fff30 2023-07-14T19:27:10.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.032+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec804e340 con 0x7faec80fff30 2023-07-14T19:27:10.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.033+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faeb8022470 con 0x7faec80fff30 2023-07-14T19:27:10.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.033+0000 7faeb77fe700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7faeb00a1260 0x7faeb00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:10.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.033+0000 7faec5d9b700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7faeb00a1260 0x7faeb00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:10.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.033+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7faeb80cdec0 con 0x7faec80fff30 2023-07-14T19:27:10.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.034+0000 7faec5d9b700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7faeb00a1260 0x7faeb00a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faebc005950 tx=0x7faebc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:10.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.037+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7faeb8097660 con 0x7faec80fff30 2023-07-14T19:27:10.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:09 smithi133 ceph-mon[128894]: pgmap v255: 97 pgs: 2 peering, 95 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.2 KiB/s wr, 20 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:10.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:09 smithi082 ceph-mon[127461]: pgmap v255: 97 pgs: 2 peering, 95 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.2 KiB/s wr, 20 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:10.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:09 smithi161 ceph-mon[129184]: pgmap v255: 97 pgs: 2 peering, 95 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.2 KiB/s wr, 20 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:10.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.274+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7faec81a2dc0 con 0x7faec80fff30 2023-07-14T19:27:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.278+0000 7faeb77fe700 1 -- 172.21.15.82:0/342608628 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260123 (secure 0 0 0) 0x7faeb809ac80 con 0x7faec80fff30 2023-07-14T19:27:10.282 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:10.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7faeb00a1260 msgr2=0x7faeb00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:10.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7faeb00a1260 0x7faeb00a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faebc005950 tx=0x7faebc0058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 msgr2=0x7faec8195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8195e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009990 tx=0x7faeb8009580 comp rx=0 tx=0).stop 2023-07-14T19:27:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 shutdown_connections 2023-07-14T19:27:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faec8104fd0 0x7faec819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7faeb00a1260 0x7faeb00a3710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faec80fff30 0x7faec8195e00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 --2- 172.21.15.82:0/342608628 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faec81008d0 0x7faec8196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 >> 172.21.15.82:0/342608628 conn(0x7faec80fb790 msgr2=0x7faec80fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 shutdown_connections 2023-07-14T19:27:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:10.281+0000 7faeccda2700 1 -- 172.21.15.82:0/342608628 wait complete. 2023-07-14T19:27:10.667 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":504215117},{"type":"v1","addr":"172.21.15.82:6801","nonce":504215117}]},"active_addr":"172.21.15.82:6801/504215117","active_change":"2023-07-14T19:21:04.644615+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.82:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1722617918}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3623265050}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3311699035}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1617843074}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3676011914}]}]} 2023-07-14T19:27:10.674 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-14T19:27:10.675 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-14T19:27:10.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T19:27:10.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:10 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/342608628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T19:27:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:10 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/342608628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T19:27:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:10 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/342608628' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-14T19:27:12.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:11 smithi133 ceph-mon[128894]: pgmap v256: 97 pgs: 97 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 13 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:12.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:11 smithi082 ceph-mon[127461]: pgmap v256: 97 pgs: 97 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 13 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:12.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:11 smithi161 ceph-mon[129184]: pgmap v256: 97 pgs: 97 active+clean; 616 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 13 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 -- 172.21.15.82:0/576594193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 msgr2=0x7f372c0fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 --2- 172.21.15.82:0/576594193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c0fe9b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f371c00aad0 tx=0x7f371c00ade0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 -- 172.21.15.82:0/576594193 shutdown_connections 2023-07-14T19:27:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 --2- 172.21.15.82:0/576594193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372c103680 0x7f372c10b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 --2- 172.21.15.82:0/576594193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c0fe9b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 --2- 172.21.15.82:0/576594193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 -- 172.21.15.82:0/576594193 >> 172.21.15.82:0/576594193 conn(0x7f372c075280 msgr2=0x7f372c075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 -- 172.21.15.82:0/576594193 shutdown_connections 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.424+0000 7f3734227700 1 -- 172.21.15.82:0/576594193 wait complete. 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 Processor -- start 2023-07-14T19:27:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 -- start start 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372c103680 0x7f372c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c19adb0 con 0x7f372c0fe5e0 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c19af20 con 0x7f372c103680 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3734227700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c19b0e0 con 0x7f372c0fef80 2023-07-14T19:27:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f37317c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f37317c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:50232/0 (socket says 172.21.15.82:50232) 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f37317c2700 1 -- 172.21.15.82:0/3052641466 learned_addr learned my addr 172.21.15.82:0/3052641466 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f3731fc3700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.425+0000 7f37327c4700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372c103680 0x7f372c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f37317c2700 1 -- 172.21.15.82:0/3052641466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372c103680 msgr2=0x7f372c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f37317c2700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372c103680 0x7f372c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f37317c2700 1 -- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 msgr2=0x7f372c195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f37317c2700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f37317c2700 1 -- 172.21.15.82:0/3052641466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c00a700 con 0x7f372c0fef80 2023-07-14T19:27:12.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f3731fc3700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:27:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f37317c2700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c196330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f372800d980 tx=0x7f372800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372800f890 con 0x7f372c0fef80 2023-07-14T19:27:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372c19b3c0 con 0x7f372c0fef80 2023-07-14T19:27:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3728010460 con 0x7f372c0fef80 2023-07-14T19:27:12.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372800e610 con 0x7f372c0fef80 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.426+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c1a2c20 con 0x7f372c0fef80 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.427+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372c04e340 con 0x7f372c0fef80 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.428+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f37280105d0 con 0x7f372c0fef80 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.429+0000 7f3722ffd700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f37180a11e0 0x7f37180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.429+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f37280cc6e0 con 0x7f372c0fef80 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.429+0000 7f3731fc3700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f37180a11e0 0x7f37180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:12.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.429+0000 7f3731fc3700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f37180a11e0 0x7f37180a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f371c000c00 tx=0x7f371c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:12.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.433+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3728095df0 con 0x7f372c0fef80 2023-07-14T19:27:12.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.623+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f372c197150 con 0x7f372c0fef80 2023-07-14T19:27:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.625+0000 7f3722ffd700 1 -- 172.21.15.82:0/3052641466 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f3728017090 con 0x7f372c0fef80 2023-07-14T19:27:12.626 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:12.626 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:27:12.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f37180a11e0 msgr2=0x7f37180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:12.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f37180a11e0 0x7f37180a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f371c000c00 tx=0x7f371c005fd0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 msgr2=0x7f372c196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:12.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c196330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f372800d980 tx=0x7f372800dc90 comp rx=0 tx=0).stop 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 shutdown_connections 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372c103680 0x7f372c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f37180a11e0 0x7f37180a3690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f372c0fe5e0 0x7f372c195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 --2- 172.21.15.82:0/3052641466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f372c0fef80 0x7f372c196330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 >> 172.21.15.82:0/3052641466 conn(0x7f372c075280 msgr2=0x7f372c0fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:12.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.627+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 shutdown_connections 2023-07-14T19:27:12.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:12.628+0000 7f3734227700 1 -- 172.21.15.82:0/3052641466 wait complete. 2023-07-14T19:27:13.082 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-14T19:27:13.082 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd dump --format=json 2023-07-14T19:27:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:13.804 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:13 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3052641466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:13.804 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:13 smithi082 ceph-mon[127461]: pgmap v257: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:13.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:13 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3052641466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:13.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:13 smithi133 ceph-mon[128894]: pgmap v257: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:13.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:13 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3052641466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:13.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:13 smithi161 ceph-mon[129184]: pgmap v257: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 14 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:14.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 -- 172.21.15.82:0/3386225874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b00fe210 msgr2=0x7f10b00fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:14.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 --2- 172.21.15.82:0/3386225874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b00fe210 0x7f10b00fe680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009990 tx=0x7f10a0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:14.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 -- 172.21.15.82:0/3386225874 shutdown_connections 2023-07-14T19:27:14.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 --2- 172.21.15.82:0/3386225874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10b00febc0 0x7f10b010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:14.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 --2- 172.21.15.82:0/3386225874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10b0104c70 0x7f10b0105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 --2- 172.21.15.82:0/3386225874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b00fe210 0x7f10b00fe680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 -- 172.21.15.82:0/3386225874 >> 172.21.15.82:0/3386225874 conn(0x7f10b0075280 msgr2=0x7f10b0075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 -- 172.21.15.82:0/3386225874 shutdown_connections 2023-07-14T19:27:14.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 -- 172.21.15.82:0/3386225874 wait complete. 2023-07-14T19:27:14.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.818+0000 7f10b65ef700 1 Processor -- start 2023-07-14T19:27:14.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 -- start start 2023-07-14T19:27:14.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10b00fe210 0x7f10b00ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:14.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10b00febc0 0x7f10b00ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:14.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 0x7f10b00ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:14.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01a4190 con 0x7f10b0104c70 2023-07-14T19:27:14.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01a4300 con 0x7f10b00fe210 2023-07-14T19:27:14.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b65ef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01a44c0 con 0x7f10b00febc0 2023-07-14T19:27:14.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 0x7f10b00ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:14.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 0x7f10b00ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40982/0 (socket says 172.21.15.82:40982) 2023-07-14T19:27:14.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 -- 172.21.15.82:0/891672915 learned_addr learned my addr 172.21.15.82:0/891672915 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:14.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10af7fe700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10b00febc0 0x7f10b00ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:14.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10affff700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10b00fe210 0x7f10b00ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:14.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 -- 172.21.15.82:0/891672915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10b00febc0 msgr2=0x7f10b00ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:14.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10b00febc0 0x7f10b00ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:14.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 -- 172.21.15.82:0/891672915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10b00fe210 msgr2=0x7f10b00ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:14.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10b00fe210 0x7f10b00ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:14.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.819+0000 7f10b4b8c700 1 -- 172.21.15.82:0/891672915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00095f0 con 0x7f10b0104c70 2023-07-14T19:27:14.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.820+0000 7f10b4b8c700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 0x7f10b00ffeb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007ee0 tx=0x7f10a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:14.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.820+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8017070 con 0x7f10b0104c70 2023-07-14T19:27:14.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.820+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b01a47a0 con 0x7f10b0104c70 2023-07-14T19:27:14.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.820+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10a800f040 con 0x7f10b0104c70 2023-07-14T19:27:14.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.820+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005580 con 0x7f10b0104c70 2023-07-14T19:27:14.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.820+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b01a4ca0 con 0x7f10b0104c70 2023-07-14T19:27:14.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.821+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b004e340 con 0x7f10b0104c70 2023-07-14T19:27:14.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.822+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f10a800db20 con 0x7f10b0104c70 2023-07-14T19:27:14.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.823+0000 7f10ad7fa700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f109c0a11e0 0x7f109c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:14.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.823+0000 7f10affff700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f109c0a11e0 0x7f109c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:14.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.823+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f10a80cc760 con 0x7f10b0104c70 2023-07-14T19:27:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.823+0000 7f10affff700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f109c0a11e0 0x7f109c0a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1098005950 tx=0x7f109800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:14.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:14.826+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f10a8095e70 con 0x7f10b0104c70 2023-07-14T19:27:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.013+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10b01a4ec0 con 0x7f10b0104c70 2023-07-14T19:27:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.015+0000 7f10ad7fa700 1 -- 172.21.15.82:0/891672915 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f10a8099490 con 0x7f10b0104c70 2023-07-14T19:27:15.016 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:15.016 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:27:15.017 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f109c0a11e0 msgr2=0x7f109c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f109c0a11e0 0x7f109c0a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1098005950 tx=0x7f109800b410 comp rx=0 tx=0).stop 2023-07-14T19:27:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 msgr2=0x7f10b00ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 0x7f10b00ffeb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007ee0 tx=0x7f10a800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 shutdown_connections 2023-07-14T19:27:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f109c0a11e0 0x7f109c0a3690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10b0104c70 0x7f10b00ffeb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10b00fe210 0x7f10b00ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 --2- 172.21.15.82:0/891672915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10b00febc0 0x7f10b00ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 >> 172.21.15.82:0/891672915 conn(0x7f10b0075280 msgr2=0x7f10b00fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 shutdown_connections 2023-07-14T19:27:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:15.017+0000 7f10b65ef700 1 -- 172.21.15.82:0/891672915 wait complete. 2023-07-14T19:27:15.387 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-14T19:27:15.388 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-14T19:27:15.388 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-14T19:27:15.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-14T19:27:15.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-14T19:27:15.390 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-14T19:27:15.391 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-14T19:27:15.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-14T19:27:15.393 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-14T19:27:15.394 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-14T19:27:15.394 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-14T19:27:15.395 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-14T19:27:16.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:15 smithi133 ceph-mon[128894]: pgmap v258: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 11 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:16.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:15 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/891672915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:16.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:15 smithi082 ceph-mon[127461]: pgmap v258: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 11 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:16.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:15 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/891672915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:16.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:15 smithi161 ceph-mon[129184]: pgmap v258: 97 pgs: 97 active+clean; 620 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.2 KiB/s wr, 11 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2023-07-14T19:27:16.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:15 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/891672915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:16.358 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:16.392 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:16.532 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:16.710 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:16.779 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:17.684 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:17.871 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:17 smithi082 ceph-mon[127461]: pgmap v259: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T19:27:18.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:17 smithi133 ceph-mon[128894]: pgmap v259: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T19:27:18.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:17 smithi161 ceph-mon[129184]: pgmap v259: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T19:27:18.496 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:18.735 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 -- 172.21.15.82:0/3154837433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 msgr2=0x7f28a010bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 --2- 172.21.15.82:0/3154837433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a010bb10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f289800b0d0 tx=0x7f289800b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 -- 172.21.15.82:0/3154837433 shutdown_connections 2023-07-14T19:27:18.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 --2- 172.21.15.82:0/3154837433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a010bb10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 --2- 172.21.15.82:0/3154837433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28a00ff580 0x7f28a00ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 --2- 172.21.15.82:0/3154837433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28a00fff20 0x7f28a0103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 -- 172.21.15.82:0/3154837433 >> 172.21.15.82:0/3154837433 conn(0x7f28a0075280 msgr2=0x7f28a0075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:18.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 -- 172.21.15.82:0/3154837433 shutdown_connections 2023-07-14T19:27:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.650+0000 7f28a6af9700 1 -- 172.21.15.82:0/3154837433 wait complete. 2023-07-14T19:27:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 Processor -- start 2023-07-14T19:27:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 -- start start 2023-07-14T19:27:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28a00ff580 0x7f28a0196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28a00fff20 0x7f28a01972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:18.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a01a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a0197bc0 con 0x7f28a0103f50 2023-07-14T19:27:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a01a2370 con 0x7f28a00ff580 2023-07-14T19:27:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a6af9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a01a24b0 con 0x7f28a00fff20 2023-07-14T19:27:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a01a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:18.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a01a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40996/0 (socket says 172.21.15.82:40996) 2023-07-14T19:27:18.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 -- 172.21.15.82:0/1505035399 learned_addr learned my addr 172.21.15.82:0/1505035399 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:18.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a4895700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28a00ff580 0x7f28a0196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:18.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 -- 172.21.15.82:0/1505035399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28a00fff20 msgr2=0x7f28a01972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:18.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28a00fff20 0x7f28a01972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 -- 172.21.15.82:0/1505035399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28a00ff580 msgr2=0x7f28a0196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:18.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28a00ff580 0x7f28a0196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 -- 172.21.15.82:0/1505035399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898009d00 con 0x7f28a0103f50 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a4895700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28a00ff580 0x7f28a0196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.651+0000 7f28a5096700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a01a1e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2898007970 tx=0x7f2898007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.652+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800e040 con 0x7f28a0103f50 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.652+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2898003e10 con 0x7f28a0103f50 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.652+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898005610 con 0x7f28a0103f50 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.652+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a01a2730 con 0x7f28a0103f50 2023-07-14T19:27:18.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.652+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a01a2af0 con 0x7f28a0103f50 2023-07-14T19:27:18.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.653+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28a004e340 con 0x7f28a0103f50 2023-07-14T19:27:18.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.654+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2898012c20 con 0x7f28a0103f50 2023-07-14T19:27:18.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.654+0000 7f289dffb700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f28900a1260 0x7f28900a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:18.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.655+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f28980ce560 con 0x7f28a0103f50 2023-07-14T19:27:18.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.655+0000 7f289dffb700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] conn(0x7f28900aac10 0x7f28900ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:18.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.655+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 --> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28900ad770 con 0x7f28900aac10 2023-07-14T19:27:18.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.655+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28980cea00 con 0x7f28a0103f50 2023-07-14T19:27:18.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.655+0000 7f289ffff700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] conn(0x7f28900aac10 0x7f28900ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:18.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.655+0000 7f289ffff700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] conn(0x7f28900aac10 0x7f28900ad0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:18.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.657+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== osd.6 v2:172.21.15.82:6818/2376027432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f28900ad770 con 0x7f28900aac10 2023-07-14T19:27:18.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.657+0000 7f28a4895700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f28900a1260 0x7f28900a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:18.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.657+0000 7f28a4895700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f28900a1260 0x7f28900a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2894005950 tx=0x7f2894005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:18.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.672+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 --> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28a010eed0 con 0x7f28900aac10 2023-07-14T19:27:18.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.672+0000 7f289dffb700 1 -- 172.21.15.82:0/1505035399 <== osd.6 v2:172.21.15.82:6818/2376027432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f288c0093c0 con 0x7f28900aac10 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] conn(0x7f28900aac10 msgr2=0x7f28900ad0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] conn(0x7f28900aac10 0x7f28900ad0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f28900a1260 msgr2=0x7f28900a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f28900a1260 0x7f28900a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2894005950 tx=0x7f2894005f50 comp rx=0 tx=0).stop 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 msgr2=0x7f28a01a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 --2- 172.21.15.82:0/1505035399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28a0103f50 0x7f28a01a1e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2898007970 tx=0x7f2898007a50 comp rx=0 tx=0).stop 2023-07-14T19:27:18.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a5096700 1 -- 172.21.15.82:0/1505035399 reap_dead start 2023-07-14T19:27:18.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 shutdown_connections 2023-07-14T19:27:18.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 >> 172.21.15.82:0/1505035399 conn(0x7f28a0075280 msgr2=0x7f28a00fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:18.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 shutdown_connections 2023-07-14T19:27:18.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:18.673+0000 7f28a6af9700 1 -- 172.21.15.82:0/1505035399 wait complete. 2023-07-14T19:27:19.174 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:19.876 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:19 smithi082 ceph-mon[127461]: pgmap v260: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T19:27:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:19 smithi133 ceph-mon[128894]: pgmap v260: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T19:27:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:19 smithi161 ceph-mon[129184]: pgmap v260: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2023-07-14T19:27:20.511 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:20.536 INFO:teuthology.orchestra.run.smithi082.stdout:193273528356 2023-07-14T19:27:20.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-14T19:27:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 -- 172.21.15.82:0/2478289493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec103ee0 msgr2=0x7facec10ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 --2- 172.21.15.82:0/2478289493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec103ee0 0x7facec10ba60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7face400b0d0 tx=0x7face400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 -- 172.21.15.82:0/2478289493 shutdown_connections 2023-07-14T19:27:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 --2- 172.21.15.82:0/2478289493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec103ee0 0x7facec10ba60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 --2- 172.21.15.82:0/2478289493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec0fee40 0x7facec0ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 --2- 172.21.15.82:0/2478289493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7facec0ff7e0 0x7facec1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 -- 172.21.15.82:0/2478289493 >> 172.21.15.82:0/2478289493 conn(0x7facec075ca0 msgr2=0x7facec0760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 -- 172.21.15.82:0/2478289493 shutdown_connections 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 -- 172.21.15.82:0/2478289493 wait complete. 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 Processor -- start 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.075+0000 7facf3c11700 1 -- start start 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec0fee40 0x7facec19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7facec0ff7e0 0x7facec19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103ee0 0x7facec194b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec1951b0 con 0x7facec0fee40 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec195320 con 0x7facec103ee0 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec1954e0 con 0x7facec0ff7e0 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec0fee40 0x7facec19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec0fee40 0x7facec19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41020/0 (socket says 172.21.15.82:41020) 2023-07-14T19:27:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 -- 172.21.15.82:0/3257197359 learned_addr learned my addr 172.21.15.82:0/3257197359 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf21ae700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103ee0 0x7facec194b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf11ac700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7facec0ff7e0 0x7facec19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 -- 172.21.15.82:0/3257197359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7facec0ff7e0 msgr2=0x7facec19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7facec0ff7e0 0x7facec19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 -- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103ee0 msgr2=0x7facec194b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103ee0 0x7facec194b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 -- 172.21.15.82:0/3257197359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009d00 con 0x7facec0fee40 2023-07-14T19:27:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf11ac700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7facec0ff7e0 0x7facec19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf19ad700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec0fee40 0x7facec19aac0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7face800d950 tx=0x7face800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8009940 con 0x7facec0fee40 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7face8010460 con 0x7facec0fee40 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec1957c0 con 0x7facec0fee40 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.076+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face800f620 con 0x7facec0fee40 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.077+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec1a1ab0 con 0x7facec0fee40 2023-07-14T19:27:21.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.077+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7facec04e340 con 0x7facec0fee40 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7face80108d0 con 0x7facec0fee40 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7face2ffd700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7facd80a11e0 0x7facd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7facf11ac700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7facd80a11e0 0x7facd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7face80ccd30 con 0x7facec0fee40 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7face2ffd700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] conn(0x7facd80aab90 0x7facd80ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 --> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facd80ad6f0 con 0x7facd80aab90 2023-07-14T19:27:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.079+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7face80cd1d0 con 0x7facec0fee40 2023-07-14T19:27:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.080+0000 7facf11ac700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7facd80a11e0 0x7facd80a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7facdc005fd0 tx=0x7facdc00a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.080+0000 7facf21ae700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] conn(0x7facd80aab90 0x7facd80ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.080+0000 7facf21ae700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] conn(0x7facd80aab90 0x7facd80ad020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.081+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== osd.7 v2:172.21.15.133:6816/3133253156 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7facd80ad6f0 con 0x7facd80aab90 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.097+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 --> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7facec10ed80 con 0x7facd80aab90 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7face2ffd700 1 -- 172.21.15.82:0/3257197359 <== osd.7 v2:172.21.15.133:6816/3133253156 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7face40093d0 con 0x7facd80aab90 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] conn(0x7facd80aab90 msgr2=0x7facd80ad020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] conn(0x7facd80aab90 0x7facd80ad020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7facd80a11e0 msgr2=0x7facd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7facd80a11e0 0x7facd80a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7facdc005fd0 tx=0x7facdc00a5e0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec0fee40 msgr2=0x7facec19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 --2- 172.21.15.82:0/3257197359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7facec0fee40 0x7facec19aac0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7face800d950 tx=0x7face800dc60 comp rx=0 tx=0).stop 2023-07-14T19:27:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf21ae700 1 -- 172.21.15.82:0/3257197359 reap_dead start 2023-07-14T19:27:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 shutdown_connections 2023-07-14T19:27:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 >> 172.21.15.82:0/3257197359 conn(0x7facec075ca0 msgr2=0x7facec102790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 shutdown_connections 2023-07-14T19:27:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:21.098+0000 7facf3c11700 1 -- 172.21.15.82:0/3257197359 wait complete. 2023-07-14T19:27:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.671+0000 7f007f6cf700 1 -- 172.21.15.82:0/4189323928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078101680 msgr2=0x7f0078109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.671+0000 7f007f6cf700 1 --2- 172.21.15.82:0/4189323928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078101680 0x7f0078109d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0068009990 tx=0x7f0068009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 -- 172.21.15.82:0/4189323928 shutdown_connections 2023-07-14T19:27:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 --2- 172.21.15.82:0/4189323928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007810a290 0x7f007810c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 --2- 172.21.15.82:0/4189323928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0078100d70 0x7f0078101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 --2- 172.21.15.82:0/4189323928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078101680 0x7f0078109d50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 -- 172.21.15.82:0/4189323928 >> 172.21.15.82:0/4189323928 conn(0x7f0078075ca0 msgr2=0x7f00780760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 -- 172.21.15.82:0/4189323928 shutdown_connections 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 -- 172.21.15.82:0/4189323928 wait complete. 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 Processor -- start 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.672+0000 7f007f6cf700 1 -- start start 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078100d70 0x7f00780feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0078101680 0x7f00780ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007810a290 0x7f00780ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a41a0 con 0x7f0078100d70 2023-07-14T19:27:21.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a4310 con 0x7f0078101680 2023-07-14T19:27:21.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781a44d0 con 0x7f007810a290 2023-07-14T19:27:21.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078100d70 0x7f00780feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078100d70 0x7f00780feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41008/0 (socket says 172.21.15.82:41008) 2023-07-14T19:27:21.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 -- 172.21.15.82:0/1318959484 learned_addr learned my addr 172.21.15.82:0/1318959484 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:21.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007cc6a700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0078101680 0x7f00780ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007dc6c700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007810a290 0x7f00780ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 -- 172.21.15.82:0/1318959484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007810a290 msgr2=0x7f00780ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f007810a290 0x7f00780ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 -- 172.21.15.82:0/1318959484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0078101680 msgr2=0x7f00780ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0078101680 0x7f00780ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 -- 172.21.15.82:0/1318959484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680095f0 con 0x7f0078100d70 2023-07-14T19:27:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007cc6a700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0078101680 0x7f00780ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007d46b700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078100d70 0x7f00780feba0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f007400d950 tx=0x7f007400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074009b40 con 0x7f0078100d70 2023-07-14T19:27:21.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0074009ca0 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400f620 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00781a47b0 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.673+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00781a4c60 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.674+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f007804e340 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.675+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0074010460 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f006e7fc700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f00640a11e0 0x7f00640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f00740cce60 con 0x7f0078100d70 2023-07-14T19:27:21.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f007cc6a700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f00640a11e0 0x7f00640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f006e7fc700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] conn(0x7f00640aab90 0x7f00640ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 --> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00640ad6f0 con 0x7f00640aab90 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00740cd300 con 0x7f0078100d70 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f007dc6c700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] conn(0x7f00640aab90 0x7f00640ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.676+0000 7f007cc6a700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f00640a11e0 0x7f00640a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0068006010 tx=0x7f006800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.677+0000 7f007dc6c700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] conn(0x7f00640aab90 0x7f00640ad020 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:21.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.677+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== osd.0 v2:172.21.15.82:6802/2396885961 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f00640ad6f0 con 0x7f00640aab90 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.693+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 --> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f007810fe70 con 0x7f00640aab90 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.693+0000 7f006e7fc700 1 -- 172.21.15.82:0/1318959484 <== osd.0 v2:172.21.15.82:6802/2396885961 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0070009c70 con 0x7f00640aab90 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] conn(0x7f00640aab90 msgr2=0x7f00640ad020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] conn(0x7f00640aab90 0x7f00640ad020 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f00640a11e0 msgr2=0x7f00640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f00640a11e0 0x7f00640a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0068006010 tx=0x7f006800b540 comp rx=0 tx=0).stop 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078100d70 msgr2=0x7f00780feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:21.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 --2- 172.21.15.82:0/1318959484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0078100d70 0x7f00780feba0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f007400d950 tx=0x7f007400dd10 comp rx=0 tx=0).stop 2023-07-14T19:27:21.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007dc6c700 1 -- 172.21.15.82:0/1318959484 reap_dead start 2023-07-14T19:27:21.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 shutdown_connections 2023-07-14T19:27:21.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 >> 172.21.15.82:0/1318959484 conn(0x7f0078075ca0 msgr2=0x7f00780fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:21.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 shutdown_connections 2023-07-14T19:27:21.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:20.694+0000 7f007f6cf700 1 -- 172.21.15.82:0/1318959484 wait complete. 2023-07-14T19:27:21.915 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:21 smithi082 ceph-mon[127461]: pgmap v261: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T19:27:22.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:21 smithi133 ceph-mon[128894]: pgmap v261: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T19:27:22.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:21 smithi161 ceph-mon[129184]: pgmap v261: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T19:27:22.259 INFO:teuthology.orchestra.run.smithi082.stdout:214748364833 2023-07-14T19:27:22.260 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-14T19:27:22.333 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:22.614 INFO:teuthology.orchestra.run.smithi082.stdout:34359738430 2023-07-14T19:27:22.615 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-14T19:27:23.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 -- 172.21.15.82:0/1907233273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 msgr2=0x7f6fc81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/1907233273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc81012f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009990 tx=0x7f6fb8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 -- 172.21.15.82:0/1907233273 shutdown_connections 2023-07-14T19:27:23.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/1907233273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fc810a270 0x7f6fc810c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/1907233273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc81012f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/1907233273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fc8101830 0x7f6fc8109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 -- 172.21.15.82:0/1907233273 >> 172.21.15.82:0/1907233273 conn(0x7f6fc8075ca0 msgr2=0x7f6fc80760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:23.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 -- 172.21.15.82:0/1907233273 shutdown_connections 2023-07-14T19:27:23.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.350+0000 7f6fce2f0700 1 -- 172.21.15.82:0/1907233273 wait complete. 2023-07-14T19:27:23.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 Processor -- start 2023-07-14T19:27:23.354 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 -- start start 2023-07-14T19:27:23.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc8195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fc8101830 0x7f6fc8196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fc810a270 0x7f6fc819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc819ad10 con 0x7f6fc8100f20 2023-07-14T19:27:23.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc819ae80 con 0x7f6fc810a270 2023-07-14T19:27:23.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.351+0000 7f6fce2f0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc819b040 con 0x7f6fc8101830 2023-07-14T19:27:23.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc8195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc8195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41034/0 (socket says 172.21.15.82:41034) 2023-07-14T19:27:23.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 -- 172.21.15.82:0/2718057195 learned_addr learned my addr 172.21.15.82:0/2718057195 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:23.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc77fe700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fc8101830 0x7f6fc8196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fcc88d700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fc810a270 0x7f6fc819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 -- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fc8101830 msgr2=0x7f6fc8196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fc8101830 0x7f6fc8196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 -- 172.21.15.82:0/2718057195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fc810a270 msgr2=0x7f6fc819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fc810a270 0x7f6fc819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 -- 172.21.15.82:0/2718057195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80095f0 con 0x7f6fc8100f20 2023-07-14T19:27:23.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fcc88d700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fc810a270 0x7f6fc819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:23.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc7fff700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc8195d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8000c00 tx=0x7f6fb800f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb801d070 con 0x7f6fc8100f20 2023-07-14T19:27:23.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc819b2c0 con 0x7f6fc8100f20 2023-07-14T19:27:23.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc81a2b80 con 0x7f6fc8100f20 2023-07-14T19:27:23.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fb800fc20 con 0x7f6fc8100f20 2023-07-14T19:27:23.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.352+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8021ea0 con 0x7f6fc8100f20 2023-07-14T19:27:23.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.353+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6fc804e340 con 0x7f6fc8100f20 2023-07-14T19:27:23.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.354+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6fb8021410 con 0x7f6fc8100f20 2023-07-14T19:27:23.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc57fa700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6fb00a1260 0x7f6fb00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f6fb80ce0a0 con 0x7f6fc8100f20 2023-07-14T19:27:23.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc77fe700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6fb00a1260 0x7f6fb00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc57fa700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] conn(0x7f6fb00aac10 0x7f6fb00ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 --> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fb00ad770 con 0x7f6fb00aac10 2023-07-14T19:27:23.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fb80ce540 con 0x7f6fc8100f20 2023-07-14T19:27:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fc77fe700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6fb00a1260 0x7f6fb00a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005950 tx=0x7f6fbc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.355+0000 7f6fcc88d700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] conn(0x7f6fb00aac10 0x7f6fb00ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.356+0000 7f6fcc88d700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] conn(0x7f6fb00aac10 0x7f6fb00ad0a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.357+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== osd.8 v2:172.21.15.161:6816/119023360 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6fb00ad770 con 0x7f6fb00aac10 2023-07-14T19:27:23.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.373+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 --> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6fc810fe50 con 0x7f6fb00aac10 2023-07-14T19:27:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fc57fa700 1 -- 172.21.15.82:0/2718057195 <== osd.8 v2:172.21.15.161:6816/119023360 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6fc0011040 con 0x7f6fb00aac10 2023-07-14T19:27:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] conn(0x7f6fb00aac10 msgr2=0x7f6fb00ad0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] conn(0x7f6fb00aac10 0x7f6fb00ad0a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6fb00a1260 msgr2=0x7f6fb00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6fb00a1260 0x7f6fb00a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005950 tx=0x7f6fbc0058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 msgr2=0x7f6fc8195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 --2- 172.21.15.82:0/2718057195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fc8100f20 0x7f6fc8195d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8000c00 tx=0x7f6fb800f510 comp rx=0 tx=0).stop 2023-07-14T19:27:23.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fcc88d700 1 -- 172.21.15.82:0/2718057195 reap_dead start 2023-07-14T19:27:23.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 shutdown_connections 2023-07-14T19:27:23.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 >> 172.21.15.82:0/2718057195 conn(0x7f6fc8075ca0 msgr2=0x7f6fc8101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:23.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 shutdown_connections 2023-07-14T19:27:23.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.374+0000 7f6fce2f0700 1 -- 172.21.15.82:0/2718057195 wait complete. 2023-07-14T19:27:23.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:23 smithi082 ceph-mon[127461]: pgmap v262: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-07-14T19:27:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 -- 172.21.15.82:0/119731783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 msgr2=0x7fcb8810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 --2- 172.21.15.82:0/119731783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8810cb50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb780099c0 tx=0x7fcb78009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 -- 172.21.15.82:0/119731783 shutdown_connections 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 --2- 172.21.15.82:0/119731783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8810cb50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 --2- 172.21.15.82:0/119731783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb880fff30 0x7fcb88100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 --2- 172.21.15.82:0/119731783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb881008d0 0x7fcb881048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 -- 172.21.15.82:0/119731783 >> 172.21.15.82:0/119731783 conn(0x7fcb880fb790 msgr2=0x7fcb880fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 -- 172.21.15.82:0/119731783 shutdown_connections 2023-07-14T19:27:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 -- 172.21.15.82:0/119731783 wait complete. 2023-07-14T19:27:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.918+0000 7fcb8de44700 1 Processor -- start 2023-07-14T19:27:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 -- start start 2023-07-14T19:27:23.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb880fff30 0x7fcb88195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb881008d0 0x7fcb881962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8819ad40 con 0x7fcb88104fd0 2023-07-14T19:27:23.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8819aeb0 con 0x7fcb880fff30 2023-07-14T19:27:23.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb8de44700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8819b070 con 0x7fcb881008d0 2023-07-14T19:27:23.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41086/0 (socket says 172.21.15.82:41086) 2023-07-14T19:27:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 -- 172.21.15.82:0/1535655852 learned_addr learned my addr 172.21.15.82:0/1535655852 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb877fe700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb880fff30 0x7fcb88195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb86ffd700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb881008d0 0x7fcb881962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 -- 172.21.15.82:0/1535655852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb881008d0 msgr2=0x7fcb881962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb881008d0 0x7fcb881962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 -- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb880fff30 msgr2=0x7fcb88195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb880fff30 0x7fcb88195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 -- 172.21.15.82:0/1535655852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780095f0 con 0x7fcb88104fd0 2023-07-14T19:27:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb877fe700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb880fff30 0x7fcb88195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:27:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.919+0000 7fcb87fff700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8819a730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcb780098e0 tx=0x7fcb78009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.920+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801d070 con 0x7fcb88104fd0 2023-07-14T19:27:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.920+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb78022470 con 0x7fcb88104fd0 2023-07-14T19:27:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.920+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78017670 con 0x7fcb88104fd0 2023-07-14T19:27:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.920+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8819b2f0 con 0x7fcb88104fd0 2023-07-14T19:27:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.920+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb881a29e0 con 0x7fcb88104fd0 2023-07-14T19:27:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.920+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb8810fe30 con 0x7fcb88104fd0 2023-07-14T19:27:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcb780225e0 con 0x7fcb88104fd0 2023-07-14T19:27:23.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb84ff9700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb740a1260 0x7fcb740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb877fe700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb740a1260 0x7fcb740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fcb780cdbc0 con 0x7fcb88104fd0 2023-07-14T19:27:23.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb84ff9700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] conn(0x7fcb740aac10 0x7fcb740ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 --> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb740ad770 con 0x7fcb740aac10 2023-07-14T19:27:23.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.922+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb780ce060 con 0x7fcb88104fd0 2023-07-14T19:27:23.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.923+0000 7fcb877fe700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb740a1260 0x7fcb740a3710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009480 tx=0x7fcb8000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.923+0000 7fcb86ffd700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] conn(0x7fcb740aac10 0x7fcb740ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.923+0000 7fcb86ffd700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] conn(0x7fcb740aac10 0x7fcb740ad0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:23 smithi133 ceph-mon[128894]: pgmap v262: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-07-14T19:27:23.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.924+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== osd.1 v2:172.21.15.133:6800/1413395016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcb740ad770 con 0x7fcb740aac10 2023-07-14T19:27:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 -- 172.21.15.82:0/1799528883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500ff810 msgr2=0x7fec501037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1799528883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500ff810 0x7fec501037e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0099c0 tx=0x7fec3c009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 -- 172.21.15.82:0/1799528883 shutdown_connections 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1799528883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec50103f10 0x7fec5010ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1799528883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec500fee70 0x7fec500ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1799528883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500ff810 0x7fec501037e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 -- 172.21.15.82:0/1799528883 >> 172.21.15.82:0/1799528883 conn(0x7fec50075d20 msgr2=0x7fec50076120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 -- 172.21.15.82:0/1799528883 shutdown_connections 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.653+0000 7fec56cdb700 1 -- 172.21.15.82:0/1799528883 wait complete. 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.654+0000 7fec56cdb700 1 Processor -- start 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.654+0000 7fec56cdb700 1 -- start start 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500fee70 0x7fec50194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec500ff810 0x7fec501952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec50103f10 0x7fec501996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50199cf0 con 0x7fec500fee70 2023-07-14T19:27:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50199e60 con 0x7fec50103f10 2023-07-14T19:27:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5019a020 con 0x7fec500ff810 2023-07-14T19:27:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500fee70 0x7fec50194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500fee70 0x7fec50194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41050/0 (socket says 172.21.15.82:41050) 2023-07-14T19:27:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 -- 172.21.15.82:0/1895004072 learned_addr learned my addr 172.21.15.82:0/1895004072 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec4ffff700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec500ff810 0x7fec501952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec55278700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec50103f10 0x7fec501996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 -- 172.21.15.82:0/1895004072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec500ff810 msgr2=0x7fec501952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec500ff810 0x7fec501952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 -- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec50103f10 msgr2=0x7fec501996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec50103f10 0x7fec501996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 -- 172.21.15.82:0/1895004072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0095f0 con 0x7fec500fee70 2023-07-14T19:27:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec4ffff700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec500ff810 0x7fec501952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec54a77700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500fee70 0x7fec50194d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec4400d950 tx=0x7fec4400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44009b40 con 0x7fec500fee70 2023-07-14T19:27:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec44009ca0 con 0x7fec500fee70 2023-07-14T19:27:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4400f620 con 0x7fec500fee70 2023-07-14T19:27:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.655+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5019a300 con 0x7fec500fee70 2023-07-14T19:27:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.656+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501a1b60 con 0x7fec500fee70 2023-07-14T19:27:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.656+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec5010ee00 con 0x7fec500fee70 2023-07-14T19:27:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fec44010460 con 0x7fec500fee70 2023-07-14T19:27:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4dffb700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fec400a12e0 0x7fec400a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4ffff700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fec400a12e0 0x7fec400a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fec440ccd80 con 0x7fec500fee70 2023-07-14T19:27:23.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4dffb700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] conn(0x7fec400aac90 0x7fec400ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:23.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 --> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec400ad7f0 con 0x7fec400aac90 2023-07-14T19:27:23.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.658+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec440d2330 con 0x7fec500fee70 2023-07-14T19:27:23.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.659+0000 7fec4ffff700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fec400a12e0 0x7fec400a3790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000c00 tx=0x7fec3c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.659+0000 7fec55278700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] conn(0x7fec400aac90 0x7fec400ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:23.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.659+0000 7fec55278700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] conn(0x7fec400aac90 0x7fec400ad120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:23.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.660+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== osd.4 v2:172.21.15.133:6808/3870116585 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fec400ad7f0 con 0x7fec400aac90 2023-07-14T19:27:23.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.691+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 --> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec5004e340 con 0x7fec400aac90 2023-07-14T19:27:23.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.691+0000 7fec4dffb700 1 -- 172.21.15.82:0/1895004072 <== osd.4 v2:172.21.15.133:6808/3870116585 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fec5004e340 con 0x7fec400aac90 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] conn(0x7fec400aac90 msgr2=0x7fec400ad120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] conn(0x7fec400aac90 0x7fec400ad120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fec400a12e0 msgr2=0x7fec400a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fec400a12e0 0x7fec400a3790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000c00 tx=0x7fec3c005fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500fee70 msgr2=0x7fec50194d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 --2- 172.21.15.82:0/1895004072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec500fee70 0x7fec50194d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec4400d950 tx=0x7fec4400dd10 comp rx=0 tx=0).stop 2023-07-14T19:27:23.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec55278700 1 -- 172.21.15.82:0/1895004072 reap_dead start 2023-07-14T19:27:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 shutdown_connections 2023-07-14T19:27:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 >> 172.21.15.82:0/1895004072 conn(0x7fec50075d20 msgr2=0x7fec501027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 shutdown_connections 2023-07-14T19:27:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.692+0000 7fec56cdb700 1 -- 172.21.15.82:0/1895004072 wait complete. 2023-07-14T19:27:23.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.941+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 --> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb8804e340 con 0x7fcb740aac10 2023-07-14T19:27:23.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb84ff9700 1 -- 172.21.15.82:0/1535655852 <== osd.1 v2:172.21.15.133:6800/1413395016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb8804e340 con 0x7fcb740aac10 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] conn(0x7fcb740aac10 msgr2=0x7fcb740ad0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] conn(0x7fcb740aac10 0x7fcb740ad0a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb740a1260 msgr2=0x7fcb740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fcb740a1260 0x7fcb740a3710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009480 tx=0x7fcb8000b040 comp rx=0 tx=0).stop 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 msgr2=0x7fcb8819a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 --2- 172.21.15.82:0/1535655852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb88104fd0 0x7fcb8819a730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcb780098e0 tx=0x7fcb78009580 comp rx=0 tx=0).stop 2023-07-14T19:27:23.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb87fff700 1 -- 172.21.15.82:0/1535655852 reap_dead start 2023-07-14T19:27:23.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 shutdown_connections 2023-07-14T19:27:23.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 >> 172.21.15.82:0/1535655852 conn(0x7fcb880fb790 msgr2=0x7fcb880fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:23.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 shutdown_connections 2023-07-14T19:27:23.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.942+0000 7fcb8de44700 1 -- 172.21.15.82:0/1535655852 wait complete. 2023-07-14T19:27:23.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:23 smithi161 ceph-mon[129184]: pgmap v262: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-07-14T19:27:24.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.331+0000 7f490447c700 1 -- 172.21.15.82:0/3902567923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0fe210 msgr2=0x7f48fc0fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.331+0000 7f490447c700 1 --2- 172.21.15.82:0/3902567923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0fe210 0x7f48fc0fe680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009990 tx=0x7f48ec009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.331+0000 7f490447c700 1 -- 172.21.15.82:0/3902567923 shutdown_connections 2023-07-14T19:27:24.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.331+0000 7f490447c700 1 --2- 172.21.15.82:0/3902567923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48fc0febc0 0x7f48fc10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.331+0000 7f490447c700 1 --2- 172.21.15.82:0/3902567923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48fc104c70 0x7f48fc105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.331+0000 7f490447c700 1 --2- 172.21.15.82:0/3902567923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0fe210 0x7f48fc0fe680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- 172.21.15.82:0/3902567923 >> 172.21.15.82:0/3902567923 conn(0x7f48fc075280 msgr2=0x7f48fc075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:24.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- 172.21.15.82:0/3902567923 shutdown_connections 2023-07-14T19:27:24.334 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- 172.21.15.82:0/3902567923 wait complete. 2023-07-14T19:27:24.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 Processor -- start 2023-07-14T19:27:24.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- start start 2023-07-14T19:27:24.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48fc0fe210 0x7f48fc195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0febc0 0x7f48fc196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48fc104c70 0x7f48fc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19ace0 con 0x7f48fc0febc0 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19ae50 con 0x7f48fc104c70 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f490447c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc19b010 con 0x7f48fc0fe210 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f4901a17700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0febc0 0x7f48fc196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f4901a17700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0febc0 0x7f48fc196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41098/0 (socket says 172.21.15.82:41098) 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.332+0000 7f4901a17700 1 -- 172.21.15.82:0/1148384242 learned_addr learned my addr 172.21.15.82:0/1148384242 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4902218700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48fc0fe210 0x7f48fc195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4902a19700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48fc104c70 0x7f48fc19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4901a17700 1 -- 172.21.15.82:0/1148384242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48fc0fe210 msgr2=0x7f48fc195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4901a17700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48fc0fe210 0x7f48fc195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4901a17700 1 -- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48fc104c70 msgr2=0x7f48fc19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4901a17700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48fc104c70 0x7f48fc19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4901a17700 1 -- 172.21.15.82:0/1148384242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec0095f0 con 0x7f48fc0febc0 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4902a19700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48fc104c70 0x7f48fc19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f4901a17700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0febc0 0x7f48fc196260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0095c0 tx=0x7f48ec00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:24.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec01d070 con 0x7f48fc0febc0 2023-07-14T19:27:24.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48ec00fb50 con 0x7f48fc0febc0 2023-07-14T19:27:24.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0173f0 con 0x7f48fc0febc0 2023-07-14T19:27:24.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc19b290 con 0x7f48fc0febc0 2023-07-14T19:27:24.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.333+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc1a2b50 con 0x7f48fc0febc0 2023-07-14T19:27:24.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.334+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48fc10ffc0 con 0x7f48fc0febc0 2023-07-14T19:27:24.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.335+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f48ec00fcc0 con 0x7f48fc0febc0 2023-07-14T19:27:24.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.335+0000 7f48f37fe700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48e80a1260 0x7f48e80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f4902218700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48e80a1260 0x7f48e80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f48ec0ce1d0 con 0x7f48fc0febc0 2023-07-14T19:27:24.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f48f37fe700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] conn(0x7f48e80aac10 0x7f48e80ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 --> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48e80ad770 con 0x7f48e80aac10 2023-07-14T19:27:24.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48ec0ce670 con 0x7f48fc0febc0 2023-07-14T19:27:24.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f4902218700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48e80a1260 0x7f48e80a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48f8005950 tx=0x7f48f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:24.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.336+0000 7f4902a19700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] conn(0x7f48e80aac10 0x7f48e80ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.337+0000 7f4902a19700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] conn(0x7f48e80aac10 0x7f48e80ad0a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:24.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.337+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== osd.10 v2:172.21.15.133:6824/2098424958 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f48e80ad770 con 0x7f48e80aac10 2023-07-14T19:27:24.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.353+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 --> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48fc04e340 con 0x7f48e80aac10 2023-07-14T19:27:24.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f48f37fe700 1 -- 172.21.15.82:0/1148384242 <== osd.10 v2:172.21.15.133:6824/2098424958 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f48fc04e340 con 0x7f48e80aac10 2023-07-14T19:27:24.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] conn(0x7f48e80aac10 msgr2=0x7f48e80ad0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f490447c700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] conn(0x7f48e80aac10 0x7f48e80ad0a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48e80a1260 msgr2=0x7f48e80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f490447c700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48e80a1260 0x7f48e80a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48f8005950 tx=0x7f48f800b410 comp rx=0 tx=0).stop 2023-07-14T19:27:24.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0febc0 msgr2=0x7f48fc196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.354+0000 7f490447c700 1 --2- 172.21.15.82:0/1148384242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48fc0febc0 0x7f48fc196260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0095c0 tx=0x7f48ec00f710 comp rx=0 tx=0).stop 2023-07-14T19:27:24.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.355+0000 7f4902a19700 1 -- 172.21.15.82:0/1148384242 reap_dead start 2023-07-14T19:27:24.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.355+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 shutdown_connections 2023-07-14T19:27:24.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.355+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 >> 172.21.15.82:0/1148384242 conn(0x7f48fc075280 msgr2=0x7f48fc0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:24.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.355+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 shutdown_connections 2023-07-14T19:27:24.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.355+0000 7f490447c700 1 -- 172.21.15.82:0/1148384242 wait complete. 2023-07-14T19:27:24.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 -- 172.21.15.82:0/1253512043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa580103680 msgr2=0x7fa58010b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 --2- 172.21.15.82:0/1253512043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa580103680 0x7fa58010b200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b0d0 tx=0x7fa57800b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 -- 172.21.15.82:0/1253512043 shutdown_connections 2023-07-14T19:27:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 --2- 172.21.15.82:0/1253512043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa580103680 0x7fa58010b200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 --2- 172.21.15.82:0/1253512043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5800fe5e0 0x7fa5800fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 --2- 172.21.15.82:0/1253512043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5800fef80 0x7fa580102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 -- 172.21.15.82:0/1253512043 >> 172.21.15.82:0/1253512043 conn(0x7fa580075280 msgr2=0x7fa580075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:24.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 -- 172.21.15.82:0/1253512043 shutdown_connections 2023-07-14T19:27:24.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.741+0000 7fa587f99700 1 -- 172.21.15.82:0/1253512043 wait complete. 2023-07-14T19:27:24.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 Processor -- start 2023-07-14T19:27:24.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 -- start start 2023-07-14T19:27:24.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5800fe5e0 0x7fa580194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5800fef80 0x7fa580195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa580103680 0x7fa580199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580199ea0 con 0x7fa5800fe5e0 2023-07-14T19:27:24.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58019a010 con 0x7fa580103680 2023-07-14T19:27:24.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa587f99700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58019a1d0 con 0x7fa5800fef80 2023-07-14T19:27:24.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5800fe5e0 0x7fa580194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5800fe5e0 0x7fa580194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41066/0 (socket says 172.21.15.82:41066) 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 -- 172.21.15.82:0/699910980 learned_addr learned my addr 172.21.15.82:0/699910980 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa586536700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa580103680 0x7fa580199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585534700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5800fef80 0x7fa580195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 -- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5800fef80 msgr2=0x7fa580195420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5800fef80 0x7fa580195420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 -- 172.21.15.82:0/699910980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa580103680 msgr2=0x7fa580199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa580103680 0x7fa580199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 -- 172.21.15.82:0/699910980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa578009d00 con 0x7fa5800fe5e0 2023-07-14T19:27:24.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585534700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5800fef80 0x7fa580195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:24.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa585d35700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5800fe5e0 0x7fa580194ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b530 tx=0x7fa57000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:24.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.742+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570011830 con 0x7fa5800fe5e0 2023-07-14T19:27:24.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.743+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa570011e50 con 0x7fa5800fe5e0 2023-07-14T19:27:24.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.743+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58019a4b0 con 0x7fa5800fe5e0 2023-07-14T19:27:24.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.743+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000f2e0 con 0x7fa5800fe5e0 2023-07-14T19:27:24.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.743+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5801a1d10 con 0x7fa5800fe5e0 2023-07-14T19:27:24.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.743+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa58004e340 con 0x7fa5800fe5e0 2023-07-14T19:27:24.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.745+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa570010380 con 0x7fa5800fe5e0 2023-07-14T19:27:24.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.745+0000 7fa576ffd700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa56c0a11e0 0x7fa56c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa585534700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa56c0a11e0 0x7fa56c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fa5700cca20 con 0x7fa5800fe5e0 2023-07-14T19:27:24.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa576ffd700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] conn(0x7fa56c0aab90 0x7fa56c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:24.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 --> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa56c0ad6f0 con 0x7fa56c0aab90 2023-07-14T19:27:24.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5700ccec0 con 0x7fa5800fe5e0 2023-07-14T19:27:24.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa585534700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa56c0a11e0 0x7fa56c0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa580196500 tx=0x7fa57c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:24.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.746+0000 7fa586536700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] conn(0x7fa56c0aab90 0x7fa56c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:24.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.747+0000 7fa586536700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] conn(0x7fa56c0aab90 0x7fa56c0ad020 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.747+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== osd.11 v2:172.21.15.161:6824/2831052737 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa56c0ad6f0 con 0x7fa56c0aab90 2023-07-14T19:27:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.764+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 --> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa58010e620 con 0x7fa56c0aab90 2023-07-14T19:27:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.765+0000 7fa576ffd700 1 -- 172.21.15.82:0/699910980 <== osd.11 v2:172.21.15.161:6824/2831052737 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa5780090d0 con 0x7fa56c0aab90 2023-07-14T19:27:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] conn(0x7fa56c0aab90 msgr2=0x7fa56c0ad020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] conn(0x7fa56c0aab90 0x7fa56c0ad020 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:24.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa56c0a11e0 msgr2=0x7fa56c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa56c0a11e0 0x7fa56c0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa580196500 tx=0x7fa57c008040 comp rx=0 tx=0).stop 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5800fe5e0 msgr2=0x7fa580194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 --2- 172.21.15.82:0/699910980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5800fe5e0 0x7fa580194ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b530 tx=0x7fa57000b840 comp rx=0 tx=0).stop 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa586536700 1 -- 172.21.15.82:0/699910980 reap_dead start 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 shutdown_connections 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 >> 172.21.15.82:0/699910980 conn(0x7fa580075280 msgr2=0x7fa580101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:24.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 shutdown_connections 2023-07-14T19:27:24.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:23.766+0000 7fa587f99700 1 -- 172.21.15.82:0/699910980 wait complete. 2023-07-14T19:27:25.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 -- 172.21.15.82:0/1577134332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac103f60 msgr2=0x7efeac10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:25.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1577134332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac103f60 0x7efeac10bae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efea400b0d0 tx=0x7efea400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 -- 172.21.15.82:0/1577134332 shutdown_connections 2023-07-14T19:27:25.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1577134332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac103f60 0x7efeac10bae0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1577134332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac0feec0 0x7efeac0ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1577134332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac0ff860 0x7efeac103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 -- 172.21.15.82:0/1577134332 >> 172.21.15.82:0/1577134332 conn(0x7efeac0fa720 msgr2=0x7efeac0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:25.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 -- 172.21.15.82:0/1577134332 shutdown_connections 2023-07-14T19:27:25.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.498+0000 7efeb09ee700 1 -- 172.21.15.82:0/1577134332 wait complete. 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 Processor -- start 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 -- start start 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac0feec0 0x7efeac196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac0ff860 0x7efeac197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac103f60 0x7efeac1a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac197b40 con 0x7efeac0feec0 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1a22b0 con 0x7efeac103f60 2023-07-14T19:27:25.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeb09ee700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac1a2420 con 0x7efeac0ff860 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac0feec0 0x7efeac196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac0feec0 0x7efeac196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41112/0 (socket says 172.21.15.82:41112) 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 -- 172.21.15.82:0/1737083495 learned_addr learned my addr 172.21.15.82:0/1737083495 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaad9d700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac103f60 0x7efeac1a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efea9d9b700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac0ff860 0x7efeac197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 -- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac0ff860 msgr2=0x7efeac197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeac0ff860 0x7efeac197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 -- 172.21.15.82:0/1737083495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac103f60 msgr2=0x7efeac1a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac103f60 0x7efeac1a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaa59c700 1 -- 172.21.15.82:0/1737083495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4009d00 con 0x7efeac0feec0 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.499+0000 7efeaad9d700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeac103f60 0x7efeac1a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efeaa59c700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac0feec0 0x7efeac196cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d950 tx=0x7efe9c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c009b40 con 0x7efeac0feec0 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe9c009ca0 con 0x7efeac0feec0 2023-07-14T19:27:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac1a2700 con 0x7efeac0feec0 2023-07-14T19:27:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00f620 con 0x7efeac0feec0 2023-07-14T19:27:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac1a2c00 con 0x7efeac0feec0 2023-07-14T19:27:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.500+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efeac0682d0 con 0x7efeac0feec0 2023-07-14T19:27:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.502+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efe9c010460 con 0x7efeac0feec0 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.502+0000 7efea37fe700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efe980a11e0 0x7efe980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.502+0000 7efea9d9b700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efe980a11e0 0x7efe980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.502+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7efe9c0cce60 con 0x7efeac0feec0 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.502+0000 7efea37fe700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] conn(0x7efe980aab90 0x7efe980ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.502+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 --> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe980ad6f0 con 0x7efe980aab90 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.503+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe9c0cd300 con 0x7efeac0feec0 2023-07-14T19:27:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.503+0000 7efea9d9b700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efe980a11e0 0x7efe980a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efeac0692d0 tx=0x7efe94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.503+0000 7efeaad9d700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] conn(0x7efe980aab90 0x7efe980ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.503+0000 7efeaad9d700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] conn(0x7efe980aab90 0x7efe980ad020 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.504+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== osd.2 v2:172.21.15.161:6800/4228439619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efe980ad6f0 con 0x7efe980aab90 2023-07-14T19:27:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.519+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 --> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efeac10ee00 con 0x7efe980aab90 2023-07-14T19:27:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efea37fe700 1 -- 172.21.15.82:0/1737083495 <== osd.2 v2:172.21.15.161:6800/4228439619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efea4009400 con 0x7efe980aab90 2023-07-14T19:27:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] conn(0x7efe980aab90 msgr2=0x7efe980ad020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] conn(0x7efe980aab90 0x7efe980ad020 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efe980a11e0 msgr2=0x7efe980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7efe980a11e0 0x7efe980a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efeac0692d0 tx=0x7efe94009260 comp rx=0 tx=0).stop 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac0feec0 msgr2=0x7efeac196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 --2- 172.21.15.82:0/1737083495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeac0feec0 0x7efeac196cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d950 tx=0x7efe9c00dd10 comp rx=0 tx=0).stop 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeaad9d700 1 -- 172.21.15.82:0/1737083495 reap_dead start 2023-07-14T19:27:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 shutdown_connections 2023-07-14T19:27:25.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 >> 172.21.15.82:0/1737083495 conn(0x7efeac0fa720 msgr2=0x7efeac0fb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:25.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 shutdown_connections 2023-07-14T19:27:25.629 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:24.520+0000 7efeb09ee700 1 -- 172.21.15.82:0/1737083495 wait complete. 2023-07-14T19:27:25.667 INFO:teuthology.orchestra.run.smithi082.stdout:55834574907 2023-07-14T19:27:25.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-14T19:27:25.854 INFO:teuthology.orchestra.run.smithi082.stdout:296352743445 2023-07-14T19:27:25.854 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-14T19:27:25.877 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:25 smithi082 ceph-mon[127461]: pgmap v263: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-07-14T19:27:26.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:25 smithi133 ceph-mon[128894]: pgmap v263: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-07-14T19:27:26.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:25 smithi161 ceph-mon[129184]: pgmap v263: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2023-07-14T19:27:26.704 INFO:teuthology.orchestra.run.smithi082.stdout:77309411382 2023-07-14T19:27:26.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-14T19:27:26.765 INFO:teuthology.orchestra.run.smithi082.stdout:141733920813 2023-07-14T19:27:26.765 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-14T19:27:27.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 -- 172.21.15.82:0/1831502756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759406d9b0 msgr2=0x7f759411b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1831502756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759406d9b0 0x7f759411b5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f758c00b0a0 tx=0x7f758c00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 -- 172.21.15.82:0/1831502756 shutdown_connections 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1831502756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759406d9b0 0x7f759411b5a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1831502756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759410e470 0x7f759406cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1831502756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f759410e8d0 0x7f759406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 -- 172.21.15.82:0/1831502756 >> 172.21.15.82:0/1831502756 conn(0x7f759406bf70 msgr2=0x7f7594071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 -- 172.21.15.82:0/1831502756 shutdown_connections 2023-07-14T19:27:27.007 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 -- 172.21.15.82:0/1831502756 wait complete. 2023-07-14T19:27:27.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.725+0000 7f759b7dc700 1 Processor -- start 2023-07-14T19:27:27.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 -- start start 2023-07-14T19:27:27.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f759406d9b0 0x7f75941aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759410e470 0x7f75941aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759410e8d0 0x7f75941a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.008 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75941a5250 con 0x7f759410e8d0 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75941a53c0 con 0x7f759410e470 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75941a5580 con 0x7f759406d9b0 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759410e8d0 0x7f75941a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759410e8d0 0x7f75941a4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41132/0 (socket says 172.21.15.82:41132) 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 -- 172.21.15.82:0/1105609190 learned_addr learned my addr 172.21.15.82:0/1105609190 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599578700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f759406d9b0 0x7f75941aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7598d77700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759410e470 0x7f75941aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 -- 172.21.15.82:0/1105609190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f759406d9b0 msgr2=0x7f75941aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f759406d9b0 0x7f75941aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 -- 172.21.15.82:0/1105609190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759410e470 msgr2=0x7f75941aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f759410e470 0x7f75941aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 -- 172.21.15.82:0/1105609190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c009d00 con 0x7f759410e8d0 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599578700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f759406d9b0 0x7f75941aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f7599d79700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759410e8d0 0x7f75941a4aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f758c0062a0 tx=0x7f758c007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00e040 con 0x7f759410e8d0 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758c004e10 con 0x7f759410e8d0 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.726+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75941a5800 con 0x7f759410e8d0 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.727+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c01d480 con 0x7f759410e8d0 2023-07-14T19:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.727+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75941b19c0 con 0x7f759410e8d0 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.727+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f759404e340 con 0x7f759410e8d0 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.729+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f758c004020 con 0x7f759410e8d0 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.729+0000 7f758a7fc700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f75800a1260 0x7f75800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.729+0000 7f7599578700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f75800a1260 0x7f75800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.729+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f758c0ceb80 con 0x7f759410e8d0 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.729+0000 7f758a7fc700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] conn(0x7f75800aac10 0x7f75800ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.729+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 --> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75800ad770 con 0x7f75800aac10 2023-07-14T19:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.730+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f758c0cf020 con 0x7f759410e8d0 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.730+0000 7f7598d77700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] conn(0x7f75800aac10 0x7f75800ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.730+0000 7f7599578700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f75800a1260 0x7f75800a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f759406dfa0 tx=0x7f7590009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.730+0000 7f7598d77700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] conn(0x7f75800aac10 0x7f75800ad0a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.731+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== osd.9 v2:172.21.15.82:6826/3744771950 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f75800ad770 con 0x7f75800aac10 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.749+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 --> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f759406fb80 con 0x7f75800aac10 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.749+0000 7f758a7fc700 1 -- 172.21.15.82:0/1105609190 <== osd.9 v2:172.21.15.82:6826/3744771950 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f759406fb80 con 0x7f75800aac10 2023-07-14T19:27:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] conn(0x7f75800aac10 msgr2=0x7f75800ad0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] conn(0x7f75800aac10 0x7f75800ad0a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f75800a1260 msgr2=0x7f75800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f75800a1260 0x7f75800a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f759406dfa0 tx=0x7f7590009670 comp rx=0 tx=0).stop 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759410e8d0 msgr2=0x7f75941a4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 --2- 172.21.15.82:0/1105609190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f759410e8d0 0x7f75941a4aa0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f758c0062a0 tx=0x7f758c007a60 comp rx=0 tx=0).stop 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f7599d79700 1 -- 172.21.15.82:0/1105609190 reap_dead start 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 shutdown_connections 2023-07-14T19:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 >> 172.21.15.82:0/1105609190 conn(0x7f759406bf70 msgr2=0x7f759406f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 shutdown_connections 2023-07-14T19:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:26.750+0000 7f759b7dc700 1 -- 172.21.15.82:0/1105609190 wait complete. 2023-07-14T19:27:27.093 INFO:teuthology.orchestra.run.smithi082.stdout:244813135901 2023-07-14T19:27:27.093 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-14T19:27:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 -- 172.21.15.82:0/158573785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa72410a270 msgr2=0x7fa72410c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 --2- 172.21.15.82:0/158573785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa72410a270 0x7fa72410c6b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00b0a0 tx=0x7fa71c00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 -- 172.21.15.82:0/158573785 shutdown_connections 2023-07-14T19:27:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 --2- 172.21.15.82:0/158573785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa72410a270 0x7fa72410c6b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 --2- 172.21.15.82:0/158573785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa724100f20 0x7fa7241012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 --2- 172.21.15.82:0/158573785 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa724101830 0x7fa724109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 -- 172.21.15.82:0/158573785 >> 172.21.15.82:0/158573785 conn(0x7fa724075ca0 msgr2=0x7fa7240760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 -- 172.21.15.82:0/158573785 shutdown_connections 2023-07-14T19:27:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.227+0000 7fa7297b2700 1 -- 172.21.15.82:0/158573785 wait complete. 2023-07-14T19:27:27.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 Processor -- start 2023-07-14T19:27:27.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 -- start start 2023-07-14T19:27:27.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa724100f20 0x7fa724195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa724101830 0x7fa724196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72410a270 0x7fa72419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419ae10 con 0x7fa724101830 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419af80 con 0x7fa72410a270 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7297b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419b140 con 0x7fa724100f20 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7227fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa724101830 0x7fa724196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7227fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa724101830 0x7fa724196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41146/0 (socket says 172.21.15.82:41146) 2023-07-14T19:27:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa7227fc700 1 -- 172.21.15.82:0/70370691 learned_addr learned my addr 172.21.15.82:0/70370691 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.228+0000 7fa722ffd700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa724100f20 0x7fa724195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7237fe700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72410a270 0x7fa72419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7227fc700 1 -- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa724100f20 msgr2=0x7fa724195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7227fc700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa724100f20 0x7fa724195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7227fc700 1 -- 172.21.15.82:0/70370691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72410a270 msgr2=0x7fa72419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7227fc700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72410a270 0x7fa72419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7227fc700 1 -- 172.21.15.82:0/70370691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009d00 con 0x7fa724101830 2023-07-14T19:27:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7227fc700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa724101830 0x7fa724196390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa71800d950 tx=0x7fa71800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71800f890 con 0x7fa724101830 2023-07-14T19:27:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72419b420 con 0x7fa724101830 2023-07-14T19:27:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa718010460 con 0x7fa724101830 2023-07-14T19:27:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71800e610 con 0x7fa724101830 2023-07-14T19:27:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.229+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7241a2c80 con 0x7fa724101830 2023-07-14T19:27:27.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.230+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa72410fe60 con 0x7fa724101830 2023-07-14T19:27:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.231+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa7180108d0 con 0x7fa724101830 2023-07-14T19:27:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa70bfff700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa70c0a11e0 0x7fa70c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa722ffd700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa70c0a11e0 0x7fa70c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fa7180ccc00 con 0x7fa724101830 2023-07-14T19:27:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa70bfff700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] conn(0x7fa70c0aab90 0x7fa70c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 --> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa70c0ad6f0 con 0x7fa70c0aab90 2023-07-14T19:27:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7180cd0a0 con 0x7fa724101830 2023-07-14T19:27:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa722ffd700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa70c0a11e0 0x7fa70c0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa714005950 tx=0x7fa71400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.232+0000 7fa7237fe700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] conn(0x7fa70c0aab90 0x7fa70c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.233+0000 7fa7237fe700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] conn(0x7fa70c0aab90 0x7fa70c0ad020 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.234+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== osd.5 v2:172.21.15.161:6808/4025083410 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa70c0ad6f0 con 0x7fa70c0aab90 2023-07-14T19:27:27.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.250+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 --> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa72404e340 con 0x7fa70c0aab90 2023-07-14T19:27:27.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa70bfff700 1 -- 172.21.15.82:0/70370691 <== osd.5 v2:172.21.15.161:6808/4025083410 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa72404e340 con 0x7fa70c0aab90 2023-07-14T19:27:27.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] conn(0x7fa70c0aab90 msgr2=0x7fa70c0ad020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] conn(0x7fa70c0aab90 0x7fa70c0ad020 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa70c0a11e0 msgr2=0x7fa70c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa70c0a11e0 0x7fa70c0a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa714005950 tx=0x7fa71400b410 comp rx=0 tx=0).stop 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa724101830 msgr2=0x7fa724196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 --2- 172.21.15.82:0/70370691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa724101830 0x7fa724196390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa71800d950 tx=0x7fa71800dc60 comp rx=0 tx=0).stop 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7237fe700 1 -- 172.21.15.82:0/70370691 reap_dead start 2023-07-14T19:27:27.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 shutdown_connections 2023-07-14T19:27:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.251+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 >> 172.21.15.82:0/70370691 conn(0x7fa724075ca0 msgr2=0x7fa7240fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.252+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 shutdown_connections 2023-07-14T19:27:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:27.252+0000 7fa7297b2700 1 -- 172.21.15.82:0/70370691 wait complete. 2023-07-14T19:27:27.456 INFO:teuthology.orchestra.run.smithi082.stdout:326417514513 2023-07-14T19:27:27.456 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-14T19:27:27.957 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:27 smithi082 ceph-mon[127461]: pgmap v264: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T19:27:27.986 INFO:teuthology.orchestra.run.smithi082.stdout:163208757291 2023-07-14T19:27:27.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-14T19:27:27.992 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:28.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:27 smithi133 ceph-mon[128894]: pgmap v264: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T19:27:28.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:27 smithi161 ceph-mon[129184]: pgmap v264: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T19:27:28.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 -- 172.21.15.82:0/2838143719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb344103ee0 msgr2=0x7fb34410ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:28.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 --2- 172.21.15.82:0/2838143719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb344103ee0 0x7fb34410ba60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b0d0 tx=0x7fb33c00b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 -- 172.21.15.82:0/2838143719 shutdown_connections 2023-07-14T19:27:28.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 --2- 172.21.15.82:0/2838143719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb344103ee0 0x7fb34410ba60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 --2- 172.21.15.82:0/2838143719 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3440fee40 0x7fb3440ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 --2- 172.21.15.82:0/2838143719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3440ff7e0 0x7fb3441037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 -- 172.21.15.82:0/2838143719 >> 172.21.15.82:0/2838143719 conn(0x7fb344075ca0 msgr2=0x7fb3440760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:28.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 -- 172.21.15.82:0/2838143719 shutdown_connections 2023-07-14T19:27:28.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.266+0000 7fb34ace7700 1 -- 172.21.15.82:0/2838143719 wait complete. 2023-07-14T19:27:28.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 Processor -- start 2023-07-14T19:27:28.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 -- start start 2023-07-14T19:27:28.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3440fee40 0x7fb344195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:28.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3440ff7e0 0x7fb344196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:28.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb344103ee0 0x7fb34419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:28.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34419aed0 con 0x7fb3440ff7e0 2023-07-14T19:27:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34419b040 con 0x7fb344103ee0 2023-07-14T19:27:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb34ace7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34419b200 con 0x7fb3440fee40 2023-07-14T19:27:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3440ff7e0 0x7fb344196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:28.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3440ff7e0 0x7fb344196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41160/0 (socket says 172.21.15.82:41160) 2023-07-14T19:27:28.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 -- 172.21.15.82:0/4121335313 learned_addr learned my addr 172.21.15.82:0/4121335313 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:28.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb349284700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb344103ee0 0x7fb34419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:28.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 -- 172.21.15.82:0/4121335313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3440fee40 msgr2=0x7fb344195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:28.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb348a83700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3440fee40 0x7fb344195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:28.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3440fee40 0x7fb344195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 -- 172.21.15.82:0/4121335313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb344103ee0 msgr2=0x7fb34419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:28.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb344103ee0 0x7fb34419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 -- 172.21.15.82:0/4121335313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c009d00 con 0x7fb3440ff7e0 2023-07-14T19:27:28.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb349284700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb344103ee0 0x7fb34419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.267+0000 7fb343fff700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3440ff7e0 0x7fb344196450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e970 tx=0x7fb33000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.268+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009790 con 0x7fb3440ff7e0 2023-07-14T19:27:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.268+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb330004cf0 con 0x7fb3440ff7e0 2023-07-14T19:27:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.268+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34419b4e0 con 0x7fb3440ff7e0 2023-07-14T19:27:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.268+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330005680 con 0x7fb3440ff7e0 2023-07-14T19:27:28.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.268+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3441a2d40 con 0x7fb3440ff7e0 2023-07-14T19:27:28.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.269+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb34410ed40 con 0x7fb3440ff7e0 2023-07-14T19:27:28.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.270+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb330004750 con 0x7fb3440ff7e0 2023-07-14T19:27:28.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.270+0000 7fb341ffb700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb3340a11e0 0x7fb3340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:28.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.270+0000 7fb348a83700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb3340a11e0 0x7fb3340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:28.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.270+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fb330014070 con 0x7fb3440ff7e0 2023-07-14T19:27:28.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.270+0000 7fb341ffb700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] conn(0x7fb3340aab80 0x7fb3340ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:28.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.271+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 --> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3340ad6e0 con 0x7fb3340aab80 2023-07-14T19:27:28.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.271+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3300ceea0 con 0x7fb3440ff7e0 2023-07-14T19:27:28.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.271+0000 7fb349284700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] conn(0x7fb3340aab80 0x7fb3340ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:28.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.271+0000 7fb348a83700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb3340a11e0 0x7fb3340a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb338005950 tx=0x7fb3380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:28.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.271+0000 7fb349284700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] conn(0x7fb3340aab80 0x7fb3340ad010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:28.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.272+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== osd.3 v2:172.21.15.82:6810/595190701 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb3340ad6e0 con 0x7fb3340aab80 2023-07-14T19:27:28.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.287+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 --> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb34404e340 con 0x7fb3340aab80 2023-07-14T19:27:28.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.287+0000 7fb341ffb700 1 -- 172.21.15.82:0/4121335313 <== osd.3 v2:172.21.15.82:6810/595190701 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb34404e340 con 0x7fb3340aab80 2023-07-14T19:27:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.287+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] conn(0x7fb3340aab80 msgr2=0x7fb3340ad010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.287+0000 7fb34ace7700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] conn(0x7fb3340aab80 0x7fb3340ad010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb3340a11e0 msgr2=0x7fb3340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb3340a11e0 0x7fb3340a3690 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb338005950 tx=0x7fb3380058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:28.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3440ff7e0 msgr2=0x7fb344196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 --2- 172.21.15.82:0/4121335313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3440ff7e0 0x7fb344196450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e970 tx=0x7fb33000ed30 comp rx=0 tx=0).stop 2023-07-14T19:27:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb349284700 1 -- 172.21.15.82:0/4121335313 reap_dead start 2023-07-14T19:27:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 shutdown_connections 2023-07-14T19:27:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 >> 172.21.15.82:0/4121335313 conn(0x7fb344075ca0 msgr2=0x7fb344102990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 shutdown_connections 2023-07-14T19:27:28.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:28.288+0000 7fb34ace7700 1 -- 172.21.15.82:0/4121335313 wait complete. 2023-07-14T19:27:28.719 INFO:teuthology.orchestra.run.smithi082.stdout:274877906970 2023-07-14T19:27:28.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-14T19:27:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:28.974 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:28.975 INFO:teuthology.orchestra.run.smithi082.stdout:111669149746 2023-07-14T19:27:28.975 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-14T19:27:30.069 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:29 smithi082 ceph-mon[127461]: pgmap v265: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:30.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:29 smithi133 ceph-mon[128894]: pgmap v265: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:30.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:29 smithi161 ceph-mon[129184]: pgmap v265: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:30.296 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:31.723 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:31.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 -- 172.21.15.82:0/2455085376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 msgr2=0x7f6c34100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:31.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 --2- 172.21.15.82:0/2455085376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34100c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c240099c0 tx=0x7f6c24009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:31.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 -- 172.21.15.82:0/2455085376 shutdown_connections 2023-07-14T19:27:31.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 --2- 172.21.15.82:0/2455085376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c34101150 0x7f6c3410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:31.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 --2- 172.21.15.82:0/2455085376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c340fef40 0x7f6c340ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:31.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 --2- 172.21.15.82:0/2455085376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34100c10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:31.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 -- 172.21.15.82:0/2455085376 >> 172.21.15.82:0/2455085376 conn(0x7f6c34075c90 msgr2=0x7f6c34076090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:31.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 -- 172.21.15.82:0/2455085376 shutdown_connections 2023-07-14T19:27:31.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 -- 172.21.15.82:0/2455085376 wait complete. 2023-07-14T19:27:31.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 Processor -- start 2023-07-14T19:27:31.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.835+0000 7f6c3b509700 1 -- start start 2023-07-14T19:27:31.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c340fef40 0x7f6c34196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:31.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:31.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c34101150 0x7f6c341a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:31.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34197ab0 con 0x7f6c341007a0 2023-07-14T19:27:31.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a21f0 con 0x7f6c34101150 2023-07-14T19:27:31.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a23b0 con 0x7f6c340fef40 2023-07-14T19:27:31.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:31.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41174/0 (socket says 172.21.15.82:41174) 2023-07-14T19:27:31.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 -- 172.21.15.82:0/509245162 learned_addr learned my addr 172.21.15.82:0/509245162 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:31.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c392a5700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c340fef40 0x7f6c34196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:31.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c39aa6700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c34101150 0x7f6c341a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:31.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 -- 172.21.15.82:0/509245162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c340fef40 msgr2=0x7f6c34196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:31.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c340fef40 0x7f6c34196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:31.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 -- 172.21.15.82:0/509245162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c34101150 msgr2=0x7f6c341a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c34101150 0x7f6c341a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 -- 172.21.15.82:0/509245162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240095f0 con 0x7f6c341007a0 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c38aa4700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34197170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009990 tx=0x7f6c24009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2401d070 con 0x7f6c341007a0 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c24004cb0 con 0x7f6c341007a0 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c341a2630 con 0x7f6c341007a0 2023-07-14T19:27:31.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24003b30 con 0x7f6c341007a0 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.836+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c341a2a40 con 0x7f6c341007a0 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.837+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3404e340 con 0x7f6c341007a0 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.838+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6c240047f0 con 0x7f6c341007a0 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.839+0000 7f6c2a7fc700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6c200a11e0 0x7f6c200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.839+0000 7f6c392a5700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6c200a11e0 0x7f6c200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.839+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f6c240cdca0 con 0x7f6c341007a0 2023-07-14T19:27:31.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.839+0000 7f6c392a5700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6c200a11e0 0x7f6c200a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005950 tx=0x7f6c3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:31.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:31.842+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6c24097340 con 0x7f6c341007a0 2023-07-14T19:27:31.908 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:31.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:31 smithi082 ceph-mon[127461]: pgmap v266: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:32.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.057+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c341a2c60 con 0x7f6c341007a0 2023-07-14T19:27:32.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.057+0000 7f6c2a7fc700 1 -- 172.21.15.82:0/509245162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c2409a960 con 0x7f6c341007a0 2023-07-14T19:27:32.059 INFO:teuthology.orchestra.run.smithi082.stdout:34359738432 2023-07-14T19:27:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6c200a11e0 msgr2=0x7f6c200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6c200a11e0 0x7f6c200a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005950 tx=0x7f6c3000b410 comp rx=0 tx=0).stop 2023-07-14T19:27:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 msgr2=0x7f6c34197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34197170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009990 tx=0x7f6c24009580 comp rx=0 tx=0).stop 2023-07-14T19:27:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 shutdown_connections 2023-07-14T19:27:32.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c34101150 0x7f6c341a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c340fef40 0x7f6c34196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6c200a11e0 0x7f6c200a3690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 --2- 172.21.15.82:0/509245162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c341007a0 0x7f6c34197170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 >> 172.21.15.82:0/509245162 conn(0x7f6c34075c90 msgr2=0x7f6c340fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 shutdown_connections 2023-07-14T19:27:32.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.059+0000 7f6c3b509700 1 -- 172.21.15.82:0/509245162 wait complete. 2023-07-14T19:27:32.164 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:32.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:31 smithi133 ceph-mon[128894]: pgmap v266: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:32.224 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 -- 172.21.15.82:0/1081735743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e8103680 msgr2=0x7f21e810b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 --2- 172.21.15.82:0/1081735743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e8103680 0x7f21e810b200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f21e000b0d0 tx=0x7f21e000b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 -- 172.21.15.82:0/1081735743 shutdown_connections 2023-07-14T19:27:32.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 --2- 172.21.15.82:0/1081735743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e8103680 0x7f21e810b200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 --2- 172.21.15.82:0/1081735743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fe5e0 0x7f21e80fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 --2- 172.21.15.82:0/1081735743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e80fef80 0x7f21e8102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 -- 172.21.15.82:0/1081735743 >> 172.21.15.82:0/1081735743 conn(0x7f21e8075280 msgr2=0x7f21e8075680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:32.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 -- 172.21.15.82:0/1081735743 shutdown_connections 2023-07-14T19:27:32.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.223+0000 7f21ee378700 1 -- 172.21.15.82:0/1081735743 wait complete. 2023-07-14T19:27:32.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 Processor -- start 2023-07-14T19:27:32.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 -- start start 2023-07-14T19:27:32.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 0x7f21e8195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fef80 0x7f21e81963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e8103680 0x7f21e819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e819ae50 con 0x7f21e80fe5e0 2023-07-14T19:27:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e819afc0 con 0x7f21e8103680 2023-07-14T19:27:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21ee378700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e819b180 con 0x7f21e80fef80 2023-07-14T19:27:32.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.224+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 0x7f21e8195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 0x7f21e8195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41186/0 (socket says 172.21.15.82:41186) 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 -- 172.21.15.82:0/1758683366 learned_addr learned my addr 172.21.15.82:0/1758683366 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e77fe700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fef80 0x7f21e81963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21ec915700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e8103680 0x7f21e819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 -- 172.21.15.82:0/1758683366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fef80 msgr2=0x7f21e81963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fef80 0x7f21e81963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 -- 172.21.15.82:0/1758683366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e8103680 msgr2=0x7f21e819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e8103680 0x7f21e819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 -- 172.21.15.82:0/1758683366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0009d00 con 0x7f21e80fe5e0 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21ec915700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e8103680 0x7f21e819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e77fe700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fef80 0x7f21e81963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e7fff700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 0x7f21e8195e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21d800d980 tx=0x7f21d800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d800bb60 con 0x7f21e80fe5e0 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e819b460 con 0x7f21e80fe5e0 2023-07-14T19:27:32.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21d8007640 con 0x7f21e80fe5e0 2023-07-14T19:27:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8004e90 con 0x7f21e80fe5e0 2023-07-14T19:27:32.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.225+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e81a2cc0 con 0x7f21e80fe5e0 2023-07-14T19:27:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.226+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e804e340 con 0x7f21e80fe5e0 2023-07-14T19:27:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.228+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f21d800bcc0 con 0x7f21e80fe5e0 2023-07-14T19:27:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.228+0000 7f21e57fa700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f21d00a11e0 0x7f21d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.228+0000 7f21e77fe700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f21d00a11e0 0x7f21d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.228+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f21d80cc840 con 0x7f21e80fe5e0 2023-07-14T19:27:32.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.229+0000 7f21e77fe700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f21d00a11e0 0x7f21d00a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005e10 tx=0x7f21dc005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:32.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.233+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f21d80999e0 con 0x7f21e80fe5e0 2023-07-14T19:27:32.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:31 smithi161 ceph-mon[129184]: pgmap v266: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:32.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.436+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f21e81a2ef0 con 0x7f21e80fe5e0 2023-07-14T19:27:32.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.437+0000 7f21e57fa700 1 -- 172.21.15.82:0/1758683366 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21d8016020 con 0x7f21e80fe5e0 2023-07-14T19:27:32.439 INFO:teuthology.orchestra.run.smithi082.stdout:193273528358 2023-07-14T19:27:32.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f21d00a11e0 msgr2=0x7f21d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f21d00a11e0 0x7f21d00a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005e10 tx=0x7f21dc005d60 comp rx=0 tx=0).stop 2023-07-14T19:27:32.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 msgr2=0x7f21e8195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 0x7f21e8195e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f21d800d980 tx=0x7f21d800dd40 comp rx=0 tx=0).stop 2023-07-14T19:27:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 shutdown_connections 2023-07-14T19:27:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e8103680 0x7f21e819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f21d00a11e0 0x7f21d00a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21e80fe5e0 0x7f21e8195e90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 --2- 172.21.15.82:0/1758683366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21e80fef80 0x7f21e81963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 >> 172.21.15.82:0/1758683366 conn(0x7f21e8075280 msgr2=0x7f21e80fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:32.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 shutdown_connections 2023-07-14T19:27:32.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.439+0000 7f21ee378700 1 -- 172.21.15.82:0/1758683366 wait complete. 2023-07-14T19:27:32.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 -- 172.21.15.82:0/4115201503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 msgr2=0x7f23400ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 --2- 172.21.15.82:0/4115201503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f23400ff210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f23300099c0 tx=0x7f2330009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 -- 172.21.15.82:0/4115201503 shutdown_connections 2023-07-14T19:27:32.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 --2- 172.21.15.82:0/4115201503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2340103ee0 0x7f234010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 --2- 172.21.15.82:0/4115201503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f23400ff210 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 --2- 172.21.15.82:0/4115201503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400ff7e0 0x7f23401037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 -- 172.21.15.82:0/4115201503 >> 172.21.15.82:0/4115201503 conn(0x7f2340075ca0 msgr2=0x7f23400760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:32.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 -- 172.21.15.82:0/4115201503 shutdown_connections 2023-07-14T19:27:32.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.463+0000 7f2344fbe700 1 -- 172.21.15.82:0/4115201503 wait complete. 2023-07-14T19:27:32.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 Processor -- start 2023-07-14T19:27:32.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 -- start start 2023-07-14T19:27:32.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f2340196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400ff7e0 0x7f2340197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340103ee0 0x7f23401a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2340197ab0 con 0x7f23400fee40 2023-07-14T19:27:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23401a21f0 con 0x7f2340103ee0 2023-07-14T19:27:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f2344fbe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23401a23b0 con 0x7f23400ff7e0 2023-07-14T19:27:32.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f2340196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f2340196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41200/0 (socket says 172.21.15.82:41200) 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 -- 172.21.15.82:0/3279116088 learned_addr learned my addr 172.21.15.82:0/3279116088 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233ed9d700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340103ee0 0x7f23401a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233dd9b700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400ff7e0 0x7f2340197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 -- 172.21.15.82:0/3279116088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400ff7e0 msgr2=0x7f2340197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400ff7e0 0x7f2340197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 -- 172.21.15.82:0/3279116088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340103ee0 msgr2=0x7f23401a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340103ee0 0x7f23401a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.464+0000 7f233e59c700 1 -- 172.21.15.82:0/3279116088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23300095f0 con 0x7f23400fee40 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.465+0000 7f233e59c700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f2340196c30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f233000b580 tx=0x7f233000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.465+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233001d070 con 0x7f23400fee40 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.465+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23401a2630 con 0x7f23400fee40 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.465+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23300054f0 con 0x7f23400fee40 2023-07-14T19:27:32.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.465+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2330004b30 con 0x7f23400fee40 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.465+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23401a2aa0 con 0x7f23400fee40 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.466+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234004e340 con 0x7f23400fee40 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.467+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f233000fb80 con 0x7f23400fee40 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.467+0000 7f232f7fe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f23280a1260 0x7f23280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.467+0000 7f233dd9b700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f23280a1260 0x7f23280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.468+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f23300cd720 con 0x7f23400fee40 2023-07-14T19:27:32.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.468+0000 7f233dd9b700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f23280a1260 0x7f23280a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23400689d0 tx=0x7f2334009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:32.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.472+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2330096ec0 con 0x7f23400fee40 2023-07-14T19:27:32.554 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:32.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.683+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f23401a2d60 con 0x7f23400fee40 2023-07-14T19:27:32.685 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.683+0000 7f232f7fe700 1 -- 172.21.15.82:0/3279116088 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f233009a4e0 con 0x7f23400fee40 2023-07-14T19:27:32.685 INFO:teuthology.orchestra.run.smithi082.stdout:141733920814 2023-07-14T19:27:32.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f23280a1260 msgr2=0x7f23280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f23280a1260 0x7f23280a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23400689d0 tx=0x7f2334009260 comp rx=0 tx=0).stop 2023-07-14T19:27:32.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 msgr2=0x7f2340196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:32.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f2340196c30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f233000b580 tx=0x7f233000f690 comp rx=0 tx=0).stop 2023-07-14T19:27:32.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 shutdown_connections 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2340103ee0 0x7f23401a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f23280a1260 0x7f23280a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23400fee40 0x7f2340196c30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 --2- 172.21.15.82:0/3279116088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400ff7e0 0x7f2340197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 >> 172.21.15.82:0/3279116088 conn(0x7f2340075ca0 msgr2=0x7f23400fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 shutdown_connections 2023-07-14T19:27:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:32.685+0000 7f2344fbe700 1 -- 172.21.15.82:0/3279116088 wait complete. 2023-07-14T19:27:32.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:32 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/509245162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T19:27:32.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:32 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1758683366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T19:27:32.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:32 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3279116088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T19:27:32.971 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738430 got 34359738432 for osd.0 2023-07-14T19:27:32.972 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:33.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 -- 172.21.15.82:0/4073873061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 msgr2=0x7f410c10cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:33.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 --2- 172.21.15.82:0/4073873061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c10cb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f410400b0d0 tx=0x7f410400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 -- 172.21.15.82:0/4073873061 shutdown_connections 2023-07-14T19:27:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 --2- 172.21.15.82:0/4073873061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c10cb50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 --2- 172.21.15.82:0/4073873061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c0fff30 0x7f410c100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 --2- 172.21.15.82:0/4073873061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f410c1008d0 0x7f410c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 -- 172.21.15.82:0/4073873061 >> 172.21.15.82:0/4073873061 conn(0x7f410c0fb790 msgr2=0x7f410c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 -- 172.21.15.82:0/4073873061 shutdown_connections 2023-07-14T19:27:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.073+0000 7f4111e5d700 1 -- 172.21.15.82:0/4073873061 wait complete. 2023-07-14T19:27:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 Processor -- start 2023-07-14T19:27:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 -- start start 2023-07-14T19:27:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f410c0fff30 0x7f410c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c1008d0 0x7f410c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19ae20 con 0x7f410c104fd0 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19af90 con 0x7f410c0fff30 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f4111e5d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19b150 con 0x7f410c1008d0 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c19a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41216/0 (socket says 172.21.15.82:41216) 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 -- 172.21.15.82:0/3375318783 learned_addr learned my addr 172.21.15.82:0/3375318783 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410b7fe700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f410c0fff30 0x7f410c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 -- 172.21.15.82:0/3375318783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c1008d0 msgr2=0x7f410c1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c1008d0 0x7f410c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 -- 172.21.15.82:0/3375318783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f410c0fff30 msgr2=0x7f410c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f410c0fff30 0x7f410c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.074+0000 7f410bfff700 1 -- 172.21.15.82:0/3375318783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009d00 con 0x7f410c104fd0 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.075+0000 7f410bfff700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c19a810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4104007870 tx=0x7f41040078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.075+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400e040 con 0x7f410c104fd0 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.075+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4104003e10 con 0x7f410c104fd0 2023-07-14T19:27:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.075+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410401b980 con 0x7f410c104fd0 2023-07-14T19:27:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.075+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c19b3d0 con 0x7f410c104fd0 2023-07-14T19:27:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.075+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c1a2c90 con 0x7f410c104fd0 2023-07-14T19:27:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.076+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c04e340 con 0x7f410c104fd0 2023-07-14T19:27:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.077+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f41040050d0 con 0x7f410c104fd0 2023-07-14T19:27:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.078+0000 7f4108ff9700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f40f40a1260 0x7f40f40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.078+0000 7f410b7fe700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f40f40a1260 0x7f40f40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.078+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f41040ce7a0 con 0x7f410c104fd0 2023-07-14T19:27:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.078+0000 7f410b7fe700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f40f40a1260 0x7f40f40a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4100005950 tx=0x7f410000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.081+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4104097e40 con 0x7f410c104fd0 2023-07-14T19:27:33.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:32 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/509245162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T19:27:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:32 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1758683366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T19:27:33.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:32 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3279116088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T19:27:33.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:32 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/509245162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-14T19:27:33.282 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:32 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1758683366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-14T19:27:33.283 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:32 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3279116088' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-14T19:27:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.274+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f410c1a2e20 con 0x7f410c104fd0 2023-07-14T19:27:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.274+0000 7f4108ff9700 1 -- 172.21.15.82:0/3375318783 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f410409b460 con 0x7f410c104fd0 2023-07-14T19:27:33.284 INFO:teuthology.orchestra.run.smithi082.stdout:55834574908 2023-07-14T19:27:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f40f40a1260 msgr2=0x7f40f40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f40f40a1260 0x7f40f40a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4100005950 tx=0x7f410000b410 comp rx=0 tx=0).stop 2023-07-14T19:27:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 msgr2=0x7f410c19a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c19a810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4104007870 tx=0x7f41040078a0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 shutdown_connections 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f40f40a1260 0x7f40f40a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f410c104fd0 0x7f410c19a810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f410c0fff30 0x7f410c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 --2- 172.21.15.82:0/3375318783 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f410c1008d0 0x7f410c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.276+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 >> 172.21.15.82:0/3375318783 conn(0x7f410c0fb790 msgr2=0x7f410c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.277+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 shutdown_connections 2023-07-14T19:27:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:33.277+0000 7f4111e5d700 1 -- 172.21.15.82:0/3375318783 wait complete. 2023-07-14T19:27:33.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920813 got 141733920814 for osd.4 2023-07-14T19:27:33.603 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:33.605 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528356 got 193273528358 for osd.6 2023-07-14T19:27:33.605 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:33.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:33 smithi082 ceph-mon[127461]: pgmap v267: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2023-07-14T19:27:33.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:33 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3375318783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T19:27:34.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:33 smithi133 ceph-mon[128894]: pgmap v267: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2023-07-14T19:27:34.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:33 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3375318783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T19:27:34.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:33 smithi161 ceph-mon[129184]: pgmap v267: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2023-07-14T19:27:34.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:33 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3375318783' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-14T19:27:34.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.306+0000 7f0f91409700 1 -- 172.21.15.82:0/213267812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 msgr2=0x7f0f8c104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.306+0000 7f0f91409700 1 --2- 172.21.15.82:0/213267812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 0x7f0f8c104840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0099c0 tx=0x7f0f7c009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 -- 172.21.15.82:0/213267812 shutdown_connections 2023-07-14T19:27:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 --2- 172.21.15.82:0/213267812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f8c104f70 0x7f0f8c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 --2- 172.21.15.82:0/213267812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 --2- 172.21.15.82:0/213267812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 0x7f0f8c104840 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 -- 172.21.15.82:0/213267812 >> 172.21.15.82:0/213267812 conn(0x7f0f8c0fb750 msgr2=0x7f0f8c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 -- 172.21.15.82:0/213267812 shutdown_connections 2023-07-14T19:27:34.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 -- 172.21.15.82:0/213267812 wait complete. 2023-07-14T19:27:34.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.307+0000 7f0f91409700 1 Processor -- start 2023-07-14T19:27:34.312 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 -- start start 2023-07-14T19:27:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 0x7f0f8c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f8c104f70 0x7f0f8c073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1a41f0 con 0x7f0f8c100870 2023-07-14T19:27:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1a4360 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f91409700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1a4520 con 0x7f0f8c104f70 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f8affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:37574/0 (socket says 172.21.15.82:37574) 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f8affd700 1 -- 172.21.15.82:0/475915162 learned_addr learned my addr 172.21.15.82:0/475915162 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.308+0000 7f0f8b7fe700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f8c104f70 0x7f0f8c073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.309+0000 7f0f8affd700 1 -- 172.21.15.82:0/475915162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f8c104f70 msgr2=0x7f0f8c073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.309+0000 7f0f8affd700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f8c104f70 0x7f0f8c073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.309+0000 7f0f8affd700 1 -- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 msgr2=0x7f0f8c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.309+0000 7f0f8affd700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 0x7f0f8c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f8affd700 1 -- 172.21.15.82:0/475915162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0095f0 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f8affd700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c072940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000d950 tx=0x7f0f8000dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:34.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80009b40 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c1a4800 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f80009ca0 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8000f620 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.310+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1a4d00 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.311+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c04e340 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.312+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0f80010460 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.313+0000 7f0f73fff700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0f740a11e0 0x7f0f740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.313+0000 7f0f8a7fc700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0f740a11e0 0x7f0f740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.313+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f0f800cc9a0 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.314+0000 7f0f8a7fc700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0f740a11e0 0x7f0f740a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009990 tx=0x7f0f7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.317+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0f80096030 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.523+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0f8c073e70 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.523+0000 7f0f73fff700 1 -- 172.21.15.82:0/475915162 <== mon.2 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f80020090 con 0x7f0f8c0ffed0 2023-07-14T19:27:34.525 INFO:teuthology.orchestra.run.smithi082.stdout:214748364835 2023-07-14T19:27:34.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0f740a11e0 msgr2=0x7f0f740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0f740a11e0 0x7f0f740a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009990 tx=0x7f0f7c000bc0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 msgr2=0x7f0f8c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c072940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000d950 tx=0x7f0f8000dd10 comp rx=0 tx=0).stop 2023-07-14T19:27:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 shutdown_connections 2023-07-14T19:27:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f8c104f70 0x7f0f8c073400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f8c0ffed0 0x7f0f8c072940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0f740a11e0 0x7f0f740a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 --2- 172.21.15.82:0/475915162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c100870 0x7f0f8c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 >> 172.21.15.82:0/475915162 conn(0x7f0f8c0fb750 msgr2=0x7f0f8c0fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 shutdown_connections 2023-07-14T19:27:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:34.526+0000 7f0f91409700 1 -- 172.21.15.82:0/475915162 wait complete. 2023-07-14T19:27:34.573 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:34.604 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:35.065 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:34 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/475915162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T19:27:35.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:34 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/475915162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T19:27:35.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:34 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/475915162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-14T19:27:35.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364833 got 214748364835 for osd.7 2023-07-14T19:27:35.819 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:35.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:35 smithi082 ceph-mon[127461]: pgmap v268: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:36.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574908 for osd.1 2023-07-14T19:27:36.147 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:36.153 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:36.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:35 smithi133 ceph-mon[128894]: pgmap v268: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:36.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:35 smithi161 ceph-mon[129184]: pgmap v268: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:27:36.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.398+0000 7fe953313700 1 -- 172.21.15.82:0/719220180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ff7e0 msgr2=0x7fe94c1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:36.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.398+0000 7fe953313700 1 --2- 172.21.15.82:0/719220180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ff7e0 0x7fe94c1037b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe9480099c0 tx=0x7fe948009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 -- 172.21.15.82:0/719220180 shutdown_connections 2023-07-14T19:27:36.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 --2- 172.21.15.82:0/719220180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe94c103ee0 0x7fe94c10ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 --2- 172.21.15.82:0/719220180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 0x7fe94c0ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 --2- 172.21.15.82:0/719220180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ff7e0 0x7fe94c1037b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 -- 172.21.15.82:0/719220180 >> 172.21.15.82:0/719220180 conn(0x7fe94c075ca0 msgr2=0x7fe94c0760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 -- 172.21.15.82:0/719220180 shutdown_connections 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 -- 172.21.15.82:0/719220180 wait complete. 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 Processor -- start 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 -- start start 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.399+0000 7fe953313700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 0x7fe94c195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe94c0ff7e0 0x7fe94c196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 0x7fe94c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19adf0 con 0x7fe94c103ee0 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19af60 con 0x7fe94c0ff7e0 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19b120 con 0x7fe94c0fee40 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 0x7fe94c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 0x7fe94c19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41244/0 (socket says 172.21.15.82:41244) 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 -- 172.21.15.82:0/2228276050 learned_addr learned my addr 172.21.15.82:0/2228276050 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9508ae700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe94c0ff7e0 0x7fe94c196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9510af700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 0x7fe94c195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 -- 172.21.15.82:0/2228276050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 msgr2=0x7fe94c195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 0x7fe94c195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 -- 172.21.15.82:0/2228276050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe94c0ff7e0 msgr2=0x7fe94c196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe94c0ff7e0 0x7fe94c196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 -- 172.21.15.82:0/2228276050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9480095f0 con 0x7fe94c103ee0 2023-07-14T19:27:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9510af700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 0x7fe94c195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9518b0700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 0x7fe94c19a7e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe944007f10 tx=0x7fe94400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944017070 con 0x7fe94c103ee0 2023-07-14T19:27:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94400f040 con 0x7fe94c103ee0 2023-07-14T19:27:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944005740 con 0x7fe94c103ee0 2023-07-14T19:27:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c19b400 con 0x7fe94c103ee0 2023-07-14T19:27:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.400+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c1a2c60 con 0x7fe94c103ee0 2023-07-14T19:27:36.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.401+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c04e340 con 0x7fe94c103ee0 2023-07-14T19:27:36.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.402+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe94400dc60 con 0x7fe94c103ee0 2023-07-14T19:27:36.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.403+0000 7fe9427fc700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9380a1260 0x7fe9380a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:36.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.403+0000 7fe9510af700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9380a1260 0x7fe9380a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:36.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.403+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fe9440cc060 con 0x7fe94c103ee0 2023-07-14T19:27:36.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.403+0000 7fe9510af700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9380a1260 0x7fe9380a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe93c007900 tx=0x7fe93c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:36.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.407+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe944095770 con 0x7fe94c103ee0 2023-07-14T19:27:36.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.612+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe94c1a2df0 con 0x7fe94c103ee0 2023-07-14T19:27:36.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.612+0000 7fe9427fc700 1 -- 172.21.15.82:0/2228276050 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe944098d90 con 0x7fe94c103ee0 2023-07-14T19:27:36.615 INFO:teuthology.orchestra.run.smithi082.stdout:77309411384 2023-07-14T19:27:36.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.616+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9380a1260 msgr2=0x7fe9380a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:36.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.616+0000 7fe953313700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9380a1260 0x7fe9380a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe93c007900 tx=0x7fe93c008040 comp rx=0 tx=0).stop 2023-07-14T19:27:36.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.616+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 msgr2=0x7fe94c19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:36.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.616+0000 7fe953313700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 0x7fe94c19a7e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe944007f10 tx=0x7fe94400d3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 shutdown_connections 2023-07-14T19:27:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9380a1260 0x7fe9380a3710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c103ee0 0x7fe94c19a7e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94c0fee40 0x7fe94c195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 --2- 172.21.15.82:0/2228276050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe94c0ff7e0 0x7fe94c196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 >> 172.21.15.82:0/2228276050 conn(0x7fe94c075ca0 msgr2=0x7fe94c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 shutdown_connections 2023-07-14T19:27:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:36.617+0000 7fe953313700 1 -- 172.21.15.82:0/2228276050 wait complete. 2023-07-14T19:27:36.801 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:36 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2228276050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T19:27:37.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:36 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2228276050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T19:27:37.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/41919090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ffcc0 msgr2=0x7fe9c810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/41919090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ffcc0 0x7fe9c810bb00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b0a0 tx=0x7fe9c000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/41919090 shutdown_connections 2023-07-14T19:27:37.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/41919090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ffcc0 0x7fe9c810bb00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/41919090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 0x7fe9c8105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/41919090 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9c80ff310 0x7fe9c80ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/41919090 >> 172.21.15.82:0/41919090 conn(0x7fe9c8075050 msgr2=0x7fe9c8075450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:37.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/41919090 shutdown_connections 2023-07-14T19:27:37.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/41919090 wait complete. 2023-07-14T19:27:37.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.196+0000 7fe9cd4e5700 1 Processor -- start 2023-07-14T19:27:37.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 -- start start 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 0x7fe9c8199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9c80ffcc0 0x7fe9c81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 0x7fe9c81a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81a4460 con 0x7fe9c80ff310 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81a45d0 con 0x7fe9c8104dc0 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9cd4e5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c81a4790 con 0x7fe9c80ffcc0 2023-07-14T19:27:37.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 0x7fe9c8199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 0x7fe9c8199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41254/0 (socket says 172.21.15.82:41254) 2023-07-14T19:27:37.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 -- 172.21.15.82:0/4186560977 learned_addr learned my addr 172.21.15.82:0/4186560977 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:37.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c77fe700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 0x7fe9c81a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c67fc700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9c80ffcc0 0x7fe9c81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 -- 172.21.15.82:0/4186560977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9c80ffcc0 msgr2=0x7fe9c81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9c80ffcc0 0x7fe9c81995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 -- 172.21.15.82:0/4186560977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 msgr2=0x7fe9c81a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 0x7fe9c81a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 -- 172.21.15.82:0/4186560977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0009d00 con 0x7fe9c80ff310 2023-07-14T19:27:37.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c77fe700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 0x7fe9c81a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:37.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.197+0000 7fe9c6ffd700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 0x7fe9c8199070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800e9a0 tx=0x7fe9b800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:37.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.198+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8009790 con 0x7fe9c80ff310 2023-07-14T19:27:37.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.198+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9b800cbd0 con 0x7fe9c80ff310 2023-07-14T19:27:37.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.198+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c81a4a70 con 0x7fe9c80ff310 2023-07-14T19:27:37.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.198+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c81a4f20 con 0x7fe9c80ff310 2023-07-14T19:27:37.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.198+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8018680 con 0x7fe9c80ff310 2023-07-14T19:27:37.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.199+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c8102f90 con 0x7fe9c80ff310 2023-07-14T19:27:37.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.200+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe9b800cd40 con 0x7fe9c80ff310 2023-07-14T19:27:37.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.200+0000 7fe9affff700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9b00a11e0 0x7fe9b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.200+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fe9b8014070 con 0x7fe9c80ff310 2023-07-14T19:27:37.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.200+0000 7fe9c67fc700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9b00a11e0 0x7fe9b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.201+0000 7fe9c67fc700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9b00a11e0 0x7fe9b00a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005950 tx=0x7fe9bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:37.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.204+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe9b8096770 con 0x7fe9c80ff310 2023-07-14T19:27:37.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:36 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2228276050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-14T19:27:37.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.413+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fe9c804e340 con 0x7fe9c80ff310 2023-07-14T19:27:37.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.413+0000 7fe9affff700 1 -- 172.21.15.82:0/4186560977 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fe9b8099d90 con 0x7fe9c80ff310 2023-07-14T19:27:37.415 INFO:teuthology.orchestra.run.smithi082.stdout:326417514515 2023-07-14T19:27:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9b00a11e0 msgr2=0x7fe9b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9b00a11e0 0x7fe9b00a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc005950 tx=0x7fe9bc0058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 msgr2=0x7fe9c8199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 0x7fe9c8199070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800e9a0 tx=0x7fe9b800ed60 comp rx=0 tx=0).stop 2023-07-14T19:27:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 shutdown_connections 2023-07-14T19:27:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c8104dc0 0x7fe9c81a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe9b00a11e0 0x7fe9b00a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c80ff310 0x7fe9c8199070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 --2- 172.21.15.82:0/4186560977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9c80ffcc0 0x7fe9c81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 >> 172.21.15.82:0/4186560977 conn(0x7fe9c8075050 msgr2=0x7fe9c810ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:37.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 shutdown_connections 2023-07-14T19:27:37.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.416+0000 7fe9cd4e5700 1 -- 172.21.15.82:0/4186560977 wait complete. 2023-07-14T19:27:37.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 -- 172.21.15.82:0/1104773978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 msgr2=0x7f603c10ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 --2- 172.21.15.82:0/1104773978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c10ba40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f603400b0d0 tx=0x7f603400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 -- 172.21.15.82:0/1104773978 shutdown_connections 2023-07-14T19:27:37.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 --2- 172.21.15.82:0/1104773978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c10ba40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 --2- 172.21.15.82:0/1104773978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c0fee20 0x7f603c0ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 --2- 172.21.15.82:0/1104773978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f603c0ff7c0 0x7f603c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 -- 172.21.15.82:0/1104773978 >> 172.21.15.82:0/1104773978 conn(0x7f603c0fa6a0 msgr2=0x7f603c0fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:37.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 -- 172.21.15.82:0/1104773978 shutdown_connections 2023-07-14T19:27:37.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.553+0000 7f6043fec700 1 -- 172.21.15.82:0/1104773978 wait complete. 2023-07-14T19:27:37.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 Processor -- start 2023-07-14T19:27:37.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 -- start start 2023-07-14T19:27:37.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f603c0fee20 0x7f603c100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c0ff7c0 0x7f603c100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c1a88f0 con 0x7f603c103ec0 2023-07-14T19:27:37.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c1a8a30 con 0x7f603c0ff7c0 2023-07-14T19:27:37.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6043fec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c1a8b70 con 0x7f603c0fee20 2023-07-14T19:27:37.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.554+0000 7f6042589700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c1012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41272/0 (socket says 172.21.15.82:41272) 2023-07-14T19:27:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 -- 172.21.15.82:0/15175780 learned_addr learned my addr 172.21.15.82:0/15175780 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6041587700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c0ff7c0 0x7f603c100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6041d88700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f603c0fee20 0x7f603c100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 -- 172.21.15.82:0/15175780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f603c0fee20 msgr2=0x7f603c100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f603c0fee20 0x7f603c100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 -- 172.21.15.82:0/15175780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c0ff7c0 msgr2=0x7f603c100d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c0ff7c0 0x7f603c100d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 -- 172.21.15.82:0/15175780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034009d00 con 0x7f603c103ec0 2023-07-14T19:27:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6042589700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c1012b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6034007820 tx=0x7f6034007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400e040 con 0x7f603c103ec0 2023-07-14T19:27:37.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c1a8df0 con 0x7f603c103ec0 2023-07-14T19:27:37.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6034003e10 con 0x7f603c103ec0 2023-07-14T19:27:37.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603401b9d0 con 0x7f603c103ec0 2023-07-14T19:27:37.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.555+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c1a9210 con 0x7f603c103ec0 2023-07-14T19:27:37.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.556+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c1081a0 con 0x7f603c103ec0 2023-07-14T19:27:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.557+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f60340056a0 con 0x7f603c103ec0 2023-07-14T19:27:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.558+0000 7f6032ffd700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60280a11e0 0x7f60280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.558+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f60340ce6e0 con 0x7f603c103ec0 2023-07-14T19:27:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.558+0000 7f6041d88700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60280a11e0 0x7f60280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.558+0000 7f6041d88700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60280a11e0 0x7f60280a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6038005950 tx=0x7f60380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.561+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6034097e80 con 0x7f603c103ec0 2023-07-14T19:27:37.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.756+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f603c04e340 con 0x7f603c103ec0 2023-07-14T19:27:37.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.756+0000 7f6032ffd700 1 -- 172.21.15.82:0/15175780 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f603409b4a0 con 0x7f603c103ec0 2023-07-14T19:27:37.758 INFO:teuthology.orchestra.run.smithi082.stdout:244813135903 2023-07-14T19:27:37.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60280a11e0 msgr2=0x7f60280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60280a11e0 0x7f60280a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6038005950 tx=0x7f60380058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 msgr2=0x7f603c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c1012b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6034007820 tx=0x7f6034007850 comp rx=0 tx=0).stop 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 shutdown_connections 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f60280a11e0 0x7f60280a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c103ec0 0x7f603c1012b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f603c0fee20 0x7f603c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 --2- 172.21.15.82:0/15175780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f603c0ff7c0 0x7f603c100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:37.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 >> 172.21.15.82:0/15175780 conn(0x7f603c0fa6a0 msgr2=0x7f603c068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:37.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 shutdown_connections 2023-07-14T19:27:37.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:37.759+0000 7f6043fec700 1 -- 172.21.15.82:0/15175780 wait complete. 2023-07-14T19:27:37.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411382 got 77309411384 for osd.2 2023-07-14T19:27:37.892 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:38.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:37 smithi133 ceph-mon[128894]: pgmap v269: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-07-14T19:27:38.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:37 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4186560977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T19:27:38.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:37 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/15175780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T19:27:38.199 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:37 smithi082 ceph-mon[127461]: pgmap v269: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-07-14T19:27:38.199 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:37 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4186560977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T19:27:38.200 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:37 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/15175780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T19:27:38.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.199+0000 7fa3f5b98700 1 -- 172.21.15.82:0/3285892185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 msgr2=0x7fa3f00ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.199+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/3285892185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f00ff290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00099c0 tx=0x7fa3e0009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 -- 172.21.15.82:0/3285892185 shutdown_connections 2023-07-14T19:27:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/3285892185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f0103f60 0x7fa3f010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/3285892185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f00ff290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/3285892185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 0x7fa3f0103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 -- 172.21.15.82:0/3285892185 >> 172.21.15.82:0/3285892185 conn(0x7fa3f00fa720 msgr2=0x7fa3f00fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 -- 172.21.15.82:0/3285892185 shutdown_connections 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 -- 172.21.15.82:0/3285892185 wait complete. 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.200+0000 7fa3f5b98700 1 Processor -- start 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 -- start start 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f0194d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 0x7fa3f0195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f0103f60 0x7fa3f01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:38.235 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0199cd0 con 0x7fa3f00feec0 2023-07-14T19:27:38.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0199e40 con 0x7fa3f00ff860 2023-07-14T19:27:38.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3f5b98700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f019a000 con 0x7fa3f0103f60 2023-07-14T19:27:38.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f0194d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:38.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f0194d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41282/0 (socket says 172.21.15.82:41282) 2023-07-14T19:27:38.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 -- 172.21.15.82:0/1741581095 learned_addr learned my addr 172.21.15.82:0/1741581095 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3eeffd700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 0x7fa3f0195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3effff700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f0103f60 0x7fa3f01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 -- 172.21.15.82:0/1741581095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f0103f60 msgr2=0x7fa3f01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f0103f60 0x7fa3f01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 -- 172.21.15.82:0/1741581095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 msgr2=0x7fa3f0195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 0x7fa3f0195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3ef7fe700 1 -- 172.21.15.82:0/1741581095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00095f0 con 0x7fa3f00feec0 2023-07-14T19:27:38.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.201+0000 7fa3eeffd700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 0x7fa3f0195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.202+0000 7fa3ef7fe700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f0194d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.202+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001f070 con 0x7fa3f00feec0 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.202+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f019a280 con 0x7fa3f00feec0 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.202+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e001ab50 con 0x7fa3f00feec0 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.202+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004120 con 0x7fa3f00feec0 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.202+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f0068bf0 con 0x7fa3f00feec0 2023-07-14T19:27:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.203+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f00682d0 con 0x7fa3f00feec0 2023-07-14T19:27:38.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.204+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa3e0023070 con 0x7fa3f00feec0 2023-07-14T19:27:38.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.204+0000 7fa3ecff9700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa3d80a11e0 0x7fa3d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:38.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.204+0000 7fa3eeffd700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa3d80a11e0 0x7fa3d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:38.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.204+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fa3e00cfb60 con 0x7fa3f00feec0 2023-07-14T19:27:38.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.205+0000 7fa3eeffd700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa3d80a11e0 0x7fa3d80a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005950 tx=0x7fa3e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:38.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.208+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa3e0099300 con 0x7fa3f00feec0 2023-07-14T19:27:38.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:37 smithi161 ceph-mon[129184]: pgmap v269: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-07-14T19:27:38.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:37 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4186560977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-14T19:27:38.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:37 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/15175780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-14T19:27:38.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514513 got 326417514515 for osd.11 2023-07-14T19:27:38.317 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:38.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.417+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fa3f0196150 con 0x7fa3f00feec0 2023-07-14T19:27:38.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.417+0000 7fa3ecff9700 1 -- 172.21.15.82:0/1741581095 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa3e009c920 con 0x7fa3f00feec0 2023-07-14T19:27:38.419 INFO:teuthology.orchestra.run.smithi082.stdout:296352743448 2023-07-14T19:27:38.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa3d80a11e0 msgr2=0x7fa3d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:38.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa3d80a11e0 0x7fa3d80a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4005950 tx=0x7fa3e40058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 msgr2=0x7fa3f0194d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:38.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f0194d10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0009580 comp rx=0 tx=0).stop 2023-07-14T19:27:38.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 shutdown_connections 2023-07-14T19:27:38.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3f0103f60 0x7fa3f01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa3d80a11e0 0x7fa3d80a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f00feec0 0x7fa3f0194d10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 --2- 172.21.15.82:0/1741581095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f00ff860 0x7fa3f0195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:38.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 >> 172.21.15.82:0/1741581095 conn(0x7fa3f00fa720 msgr2=0x7fa3f00fb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:38.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 shutdown_connections 2023-07-14T19:27:38.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:38.419+0000 7fa3f5b98700 1 -- 172.21.15.82:0/1741581095 wait complete. 2023-07-14T19:27:38.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:38 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1741581095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T19:27:39.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:38 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1741581095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T19:27:39.207 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743448 for osd.10 2023-07-14T19:27:39.207 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:39.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:38 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1741581095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-14T19:27:39.545 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135901 got 244813135903 for osd.8 2023-07-14T19:27:39.545 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 -- 172.21.15.82:0/3307452452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb130100f20 msgr2=0x7fb1301012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3307452452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb130100f20 0x7fb1301012f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb124009990 tx=0x7fb124009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:39.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 -- 172.21.15.82:0/3307452452 shutdown_connections 2023-07-14T19:27:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3307452452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb13010a270 0x7fb13010c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3307452452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb130100f20 0x7fb1301012f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3307452452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb130101830 0x7fb130109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:39.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 -- 172.21.15.82:0/3307452452 >> 172.21.15.82:0/3307452452 conn(0x7fb130075ca0 msgr2=0x7fb1300760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 -- 172.21.15.82:0/3307452452 shutdown_connections 2023-07-14T19:27:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.836+0000 7fb136f9b700 1 -- 172.21.15.82:0/3307452452 wait complete. 2023-07-14T19:27:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 Processor -- start 2023-07-14T19:27:39.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 -- start start 2023-07-14T19:27:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb130100f20 0x7fb130195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb130101830 0x7fb130196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 0x7fb1301995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:39.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb130199bd0 con 0x7fb13010a270 2023-07-14T19:27:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb130199d40 con 0x7fb130100f20 2023-07-14T19:27:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb136f9b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb130199f00 con 0x7fb130101830 2023-07-14T19:27:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 0x7fb1301995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 0x7fb1301995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41298/0 (socket says 172.21.15.82:41298) 2023-07-14T19:27:39.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 -- 172.21.15.82:0/3139636835 learned_addr learned my addr 172.21.15.82:0/3139636835 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb134d37700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb130100f20 0x7fb130195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb12ffff700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb130101830 0x7fb130196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 -- 172.21.15.82:0/3139636835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb130101830 msgr2=0x7fb130196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb130101830 0x7fb130196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 -- 172.21.15.82:0/3139636835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb130100f20 msgr2=0x7fb130195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb130100f20 0x7fb130195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:39.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 -- 172.21.15.82:0/3139636835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240095f0 con 0x7fb13010a270 2023-07-14T19:27:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb134d37700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb130100f20 0x7fb130195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.837+0000 7fb135538700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 0x7fb1301995d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb12800cca0 tx=0x7fb12800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.838+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800cea0 con 0x7fb13010a270 2023-07-14T19:27:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.838+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12800ad60 con 0x7fb13010a270 2023-07-14T19:27:39.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.838+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13019a1e0 con 0x7fb13010a270 2023-07-14T19:27:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.838+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1280116f0 con 0x7fb13010a270 2023-07-14T19:27:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.838+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1301a2b30 con 0x7fb13010a270 2023-07-14T19:27:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.839+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb130104ae0 con 0x7fb13010a270 2023-07-14T19:27:39.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.840+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb128012420 con 0x7fb13010a270 2023-07-14T19:27:39.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.840+0000 7fb12dffb700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb1200a11e0 0x7fb1200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:39.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.840+0000 7fb134d37700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb1200a11e0 0x7fb1200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:39.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.840+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fb1280ccaa0 con 0x7fb13010a270 2023-07-14T19:27:39.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.841+0000 7fb134d37700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb1200a11e0 0x7fb1200a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb12400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:39.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:39.844+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb1280962c0 con 0x7fb13010a270 2023-07-14T19:27:39.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:39 smithi082 ceph-mon[127461]: pgmap v270: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:27:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.042+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fb1301044c0 con 0x7fb13010a270 2023-07-14T19:27:40.044 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.042+0000 7fb12dffb700 1 -- 172.21.15.82:0/3139636835 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1280998e0 con 0x7fb13010a270 2023-07-14T19:27:40.045 INFO:teuthology.orchestra.run.smithi082.stdout:274877906972 2023-07-14T19:27:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb1200a11e0 msgr2=0x7fb1200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb1200a11e0 0x7fb1200a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb124006010 tx=0x7fb12400b580 comp rx=0 tx=0).stop 2023-07-14T19:27:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 msgr2=0x7fb1301995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.046 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 0x7fb1301995d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb12800cca0 tx=0x7fb12800a340 comp rx=0 tx=0).stop 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 shutdown_connections 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb1200a11e0 0x7fb1200a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb13010a270 0x7fb1301995d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb130100f20 0x7fb130195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.044+0000 7fb136f9b700 1 --2- 172.21.15.82:0/3139636835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb130101830 0x7fb130196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.045+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 >> 172.21.15.82:0/3139636835 conn(0x7fb130075ca0 msgr2=0x7fb130101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.045+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 shutdown_connections 2023-07-14T19:27:40.047 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.045+0000 7fb136f9b700 1 -- 172.21.15.82:0/3139636835 wait complete. 2023-07-14T19:27:40.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:39 smithi133 ceph-mon[128894]: pgmap v270: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:27:40.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 -- 172.21.15.82:0/2522064077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140ff7e0 msgr2=0x7fad141037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 --2- 172.21.15.82:0/2522064077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140ff7e0 0x7fad141037b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7facfc0099c0 tx=0x7facfc009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 -- 172.21.15.82:0/2522064077 shutdown_connections 2023-07-14T19:27:40.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 --2- 172.21.15.82:0/2522064077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14103ee0 0x7fad1410ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 --2- 172.21.15.82:0/2522064077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140fee40 0x7fad140ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 --2- 172.21.15.82:0/2522064077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140ff7e0 0x7fad141037b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 -- 172.21.15.82:0/2522064077 >> 172.21.15.82:0/2522064077 conn(0x7fad14075ca0 msgr2=0x7fad140760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:40.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 -- 172.21.15.82:0/2522064077 shutdown_connections 2023-07-14T19:27:40.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.210+0000 7fad19f22700 1 -- 172.21.15.82:0/2522064077 wait complete. 2023-07-14T19:27:40.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 Processor -- start 2023-07-14T19:27:40.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 -- start start 2023-07-14T19:27:40.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 0x7fad1419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140ff7e0 0x7fad1419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14103ee0 0x7fad14194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad141950f0 con 0x7fad140fee40 2023-07-14T19:27:40.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14195260 con 0x7fad140ff7e0 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad19f22700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14195420 con 0x7fad14103ee0 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad137fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 0x7fad1419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad137fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 0x7fad1419aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41308/0 (socket says 172.21.15.82:41308) 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad137fe700 1 -- 172.21.15.82:0/877072998 learned_addr learned my addr 172.21.15.82:0/877072998 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad12ffd700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140ff7e0 0x7fad1419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad137fe700 1 -- 172.21.15.82:0/877072998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14103ee0 msgr2=0x7fad14194a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.211+0000 7fad13fff700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14103ee0 0x7fad14194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad137fe700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14103ee0 0x7fad14194a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad137fe700 1 -- 172.21.15.82:0/877072998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140ff7e0 msgr2=0x7fad1419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad137fe700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140ff7e0 0x7fad1419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad137fe700 1 -- 172.21.15.82:0/877072998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0095f0 con 0x7fad140fee40 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad12ffd700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140ff7e0 0x7fad1419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad137fe700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 0x7fad1419aa00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c9b0 tx=0x7fad0400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04007b00 con 0x7fad140fee40 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad04007c60 con 0x7fad140fee40 2023-07-14T19:27:40.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04018660 con 0x7fad140fee40 2023-07-14T19:27:40.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad14195700 con 0x7fad140fee40 2023-07-14T19:27:40.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.212+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad140682d0 con 0x7fad140fee40 2023-07-14T19:27:40.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.213+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad141958f0 con 0x7fad140fee40 2023-07-14T19:27:40.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.214+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fad04018880 con 0x7fad140fee40 2023-07-14T19:27:40.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.215+0000 7fad10ff9700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fad000a11e0 0x7fad000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.215+0000 7fad12ffd700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fad000a11e0 0x7fad000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.215+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fad040ccb20 con 0x7fad140fee40 2023-07-14T19:27:40.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.215+0000 7fad12ffd700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fad000a11e0 0x7fad000a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.218+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fad040962c0 con 0x7fad140fee40 2023-07-14T19:27:40.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:39 smithi161 ceph-mon[129184]: pgmap v270: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:27:40.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.420+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fad141958f0 con 0x7fad140fee40 2023-07-14T19:27:40.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.420+0000 7fad10ff9700 1 -- 172.21.15.82:0/877072998 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad141958f0 con 0x7fad140fee40 2023-07-14T19:27:40.422 INFO:teuthology.orchestra.run.smithi082.stdout:111669149749 2023-07-14T19:27:40.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fad000a11e0 msgr2=0x7fad000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fad000a11e0 0x7fad000a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc005fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 msgr2=0x7fad1419aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 0x7fad1419aa00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c9b0 tx=0x7fad0400cd70 comp rx=0 tx=0).stop 2023-07-14T19:27:40.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 shutdown_connections 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad14103ee0 0x7fad14194a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fad000a11e0 0x7fad000a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140fee40 0x7fad1419aa00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 --2- 172.21.15.82:0/877072998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad140ff7e0 0x7fad1419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.422+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 >> 172.21.15.82:0/877072998 conn(0x7fad14075ca0 msgr2=0x7fad140fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.423+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 shutdown_connections 2023-07-14T19:27:40.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.423+0000 7fad19f22700 1 -- 172.21.15.82:0/877072998 wait complete. 2023-07-14T19:27:40.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 -- 172.21.15.82:0/1491746864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c0fff60 msgr2=0x7f8e5c100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 --2- 172.21.15.82:0/1491746864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c0fff60 0x7f8e5c100330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0099c0 tx=0x7f8e4c009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 -- 172.21.15.82:0/1491746864 shutdown_connections 2023-07-14T19:27:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 --2- 172.21.15.82:0/1491746864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c10cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 --2- 172.21.15.82:0/1491746864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c0fff60 0x7f8e5c100330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 --2- 172.21.15.82:0/1491746864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e5c100900 0x7f8e5c1048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 -- 172.21.15.82:0/1491746864 >> 172.21.15.82:0/1491746864 conn(0x7f8e5c077ca0 msgr2=0x7f8e5c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:40.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 -- 172.21.15.82:0/1491746864 shutdown_connections 2023-07-14T19:27:40.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 -- 172.21.15.82:0/1491746864 wait complete. 2023-07-14T19:27:40.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 Processor -- start 2023-07-14T19:27:40.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.466+0000 7f8e63f17700 1 -- start start 2023-07-14T19:27:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e5c0fff60 0x7f8e5c072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c100900 0x7f8e5c072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a4150 con 0x7f8e5c104ff0 2023-07-14T19:27:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a42c0 con 0x7f8e5c100900 2023-07-14T19:27:40.471 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a4480 con 0x7f8e5c0fff60 2023-07-14T19:27:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c073400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41326/0 (socket says 172.21.15.82:41326) 2023-07-14T19:27:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 -- 172.21.15.82:0/404913307 learned_addr learned my addr 172.21.15.82:0/404913307 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e614b2700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c100900 0x7f8e5c072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e61cb3700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e5c0fff60 0x7f8e5c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 -- 172.21.15.82:0/404913307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e5c0fff60 msgr2=0x7f8e5c072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e5c0fff60 0x7f8e5c072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 -- 172.21.15.82:0/404913307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c100900 msgr2=0x7f8e5c072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c100900 0x7f8e5c072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 -- 172.21.15.82:0/404913307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0095f0 con 0x7f8e5c104ff0 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e614b2700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c100900 0x7f8e5c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e624b4700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c073400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8e54007d70 tx=0x7f8e5400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400ad40 con 0x7f8e5c104ff0 2023-07-14T19:27:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e540128e0 con 0x7f8e5c104ff0 2023-07-14T19:27:40.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c1a4760 con 0x7f8e5c104ff0 2023-07-14T19:27:40.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.467+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54011780 con 0x7f8e5c104ff0 2023-07-14T19:27:40.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.468+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c1a4c10 con 0x7f8e5c104ff0 2023-07-14T19:27:40.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.468+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c04e340 con 0x7f8e5c104ff0 2023-07-14T19:27:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.469+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8e54012a50 con 0x7f8e5c104ff0 2023-07-14T19:27:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.470+0000 7f8e52ffd700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e480a11e0 0x7f8e480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.470+0000 7f8e61cb3700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e480a11e0 0x7f8e480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.470+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f8e540ccf40 con 0x7f8e5c104ff0 2023-07-14T19:27:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.471+0000 7f8e61cb3700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e480a11e0 0x7f8e480a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.474+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8e54096760 con 0x7f8e5c104ff0 2023-07-14T19:27:40.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.674+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8e5c1a4fc0 con 0x7f8e5c104ff0 2023-07-14T19:27:40.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.675+0000 7f8e52ffd700 1 -- 172.21.15.82:0/404913307 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e54016020 con 0x7f8e5c104ff0 2023-07-14T19:27:40.677 INFO:teuthology.orchestra.run.smithi082.stdout:163208757293 2023-07-14T19:27:40.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e480a11e0 msgr2=0x7f8e480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e480a11e0 0x7f8e480a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c005fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 msgr2=0x7f8e5c073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:40.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c073400 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8e54007d70 tx=0x7f8e5400a340 comp rx=0 tx=0).stop 2023-07-14T19:27:40.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 shutdown_connections 2023-07-14T19:27:40.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e480a11e0 0x7f8e480a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c104ff0 0x7f8e5c073400 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e5c0fff60 0x7f8e5c072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 --2- 172.21.15.82:0/404913307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e5c100900 0x7f8e5c072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 >> 172.21.15.82:0/404913307 conn(0x7f8e5c077ca0 msgr2=0x7f8e5c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 shutdown_connections 2023-07-14T19:27:40.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:40.677+0000 7f8e63f17700 1 -- 172.21.15.82:0/404913307 wait complete. 2023-07-14T19:27:40.913 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:40 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3139636835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T19:27:40.913 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:40 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/877072998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T19:27:40.913 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:40 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/404913307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T19:27:40.967 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149746 got 111669149749 for osd.3 2023-07-14T19:27:40.967 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:40 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3139636835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T19:27:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:40 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/877072998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T19:27:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:40 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/404913307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T19:27:41.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:40 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3139636835' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-14T19:27:41.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:40 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/877072998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-14T19:27:41.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:40 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/404913307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-14T19:27:41.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757291 got 163208757293 for osd.5 2023-07-14T19:27:41.363 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:41.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906972 for osd.9 2023-07-14T19:27:41.505 DEBUG:teuthology.parallel:result is None 2023-07-14T19:27:41.505 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-14T19:27:41.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T19:27:42.056 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:42.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:41 smithi133 ceph-mon[128894]: pgmap v271: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:42.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:41 smithi082 ceph-mon[127461]: pgmap v271: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:42.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:41 smithi161 ceph-mon[129184]: pgmap v271: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:43.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 -- 172.21.15.82:0/3001240129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb00fff30 msgr2=0x7f3bb0100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/3001240129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb00fff30 0x7f3bb0100300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3b980099c0 tx=0x7f3b98009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 -- 172.21.15.82:0/3001240129 shutdown_connections 2023-07-14T19:27:43.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/3001240129 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb0104fd0 0x7f3bb010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/3001240129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb00fff30 0x7f3bb0100300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/3001240129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb01008d0 0x7f3bb01048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 -- 172.21.15.82:0/3001240129 >> 172.21.15.82:0/3001240129 conn(0x7f3bb00fb790 msgr2=0x7f3bb00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:43.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 -- 172.21.15.82:0/3001240129 shutdown_connections 2023-07-14T19:27:43.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.258+0000 7f3bb67a7700 1 -- 172.21.15.82:0/3001240129 wait complete. 2023-07-14T19:27:43.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 Processor -- start 2023-07-14T19:27:43.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 -- start start 2023-07-14T19:27:43.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00fff30 0x7f3bb0195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:43.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 0x7f3bb0196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:43.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb0104fd0 0x7f3bb019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:43.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb019ad10 con 0x7f3bb01008d0 2023-07-14T19:27:43.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb019ae80 con 0x7f3bb0104fd0 2023-07-14T19:27:43.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb67a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb019b040 con 0x7f3bb00fff30 2023-07-14T19:27:43.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 0x7f3bb0196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:43.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 0x7f3bb0196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41338/0 (socket says 172.21.15.82:41338) 2023-07-14T19:27:43.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 -- 172.21.15.82:0/2842470039 learned_addr learned my addr 172.21.15.82:0/2842470039 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:43.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb4d44700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb0104fd0 0x7f3bb019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:43.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baffff700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00fff30 0x7f3bb0195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:43.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 -- 172.21.15.82:0/2842470039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00fff30 msgr2=0x7f3bb0195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:43.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00fff30 0x7f3bb0195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 -- 172.21.15.82:0/2842470039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb0104fd0 msgr2=0x7f3bb019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:43.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb0104fd0 0x7f3bb019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3baf7fe700 1 -- 172.21.15.82:0/2842470039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980095f0 con 0x7f3bb01008d0 2023-07-14T19:27:43.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.259+0000 7f3bb4d44700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb0104fd0 0x7f3bb019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:43.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.260+0000 7f3baf7fe700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 0x7f3bb0196290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000d980 tx=0x7f3ba000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:43.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.260+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0009b40 con 0x7f3bb01008d0 2023-07-14T19:27:43.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.260+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba000de60 con 0x7f3bb01008d0 2023-07-14T19:27:43.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.260+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb019b320 con 0x7f3bb01008d0 2023-07-14T19:27:43.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.260+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000f620 con 0x7f3bb01008d0 2023-07-14T19:27:43.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.260+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb0072a40 con 0x7f3bb01008d0 2023-07-14T19:27:43.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.261+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb004e340 con 0x7f3bb01008d0 2023-07-14T19:27:43.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.262+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ba0010460 con 0x7f3bb01008d0 2023-07-14T19:27:43.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.264+0000 7f3bad7fa700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3b9c0a11e0 0x7f3b9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:43.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.264+0000 7f3baffff700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3b9c0a11e0 0x7f3b9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:43.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.264+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f3ba00cc550 con 0x7f3bb01008d0 2023-07-14T19:27:43.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.264+0000 7f3baffff700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3b9c0a11e0 0x7f3b9c0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009fd0 tx=0x7f3b98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:43.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.266+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3ba0095c60 con 0x7f3bb01008d0 2023-07-14T19:27:43.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.447+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bb00fc950 con 0x7f3b9c0a11e0 2023-07-14T19:27:43.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.457+0000 7f3bad7fa700 1 -- 172.21.15.82:0/2842470039 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339934 (secure 0 0 0) 0x7f3bb00fc950 con 0x7f3b9c0a11e0 2023-07-14T19:27:43.462 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3b9c0a11e0 msgr2=0x7f3b9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3b9c0a11e0 0x7f3b9c0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009fd0 tx=0x7f3b98005fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 msgr2=0x7f3bb0196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 0x7f3bb0196290 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000d980 tx=0x7f3ba000dd40 comp rx=0 tx=0).stop 2023-07-14T19:27:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 shutdown_connections 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bb0104fd0 0x7f3bb019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bb00fff30 0x7f3bb0195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3b9c0a11e0 0x7f3b9c0a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 --2- 172.21.15.82:0/2842470039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb01008d0 0x7f3bb0196290 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 >> 172.21.15.82:0/2842470039 conn(0x7f3bb00fb790 msgr2=0x7f3bb00fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 shutdown_connections 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:43.460+0000 7f3bb67a7700 1 -- 172.21.15.82:0/2842470039 wait complete. 2023-07-14T19:27:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:27:43.849 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:43 smithi082 ceph-mon[127461]: pgmap v272: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:27:43.907 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2023-07-14T19:27:42.727915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650840,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":397,"num_read_kb":313,"num_write":330,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335216,"kb_used_data":10972,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606608,"statfs":{"total":1151940427776,"available":1151597166592,"internally_reserved":0,"allocated":11235328,"data_stored":7126162,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002776"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792664+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:27:05.792664+0000","last_peered":"2023-07-14T19:27:05.792664+0000","last_clean":"2023-07-14T19:27:05.792664+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:27:05.792664+0000","last_undegraded":"2023-07-14T19:27:05.792664+0000","last_fullsized":"2023-07-14T19:27:05.792664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:41.471339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952245+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:27:05.952245+0000","last_peered":"2023-07-14T19:27:05.952245+0000","last_clean":"2023-07-14T19:27:05.952245+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:27:05.952245+0000","last_undegraded":"2023-07-14T19:27:05.952245+0000","last_fullsized":"2023-07-14T19:27:05.952245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:01.099918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952318+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:27:05.952318+0000","last_peered":"2023-07-14T19:27:05.952318+0000","last_clean":"2023-07-14T19:27:05.952318+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:27:05.952318+0000","last_undegraded":"2023-07-14T19:27:05.952318+0000","last_fullsized":"2023-07-14T19:27:05.952318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:33:44.231618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789199+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:27:05.789199+0000","last_peered":"2023-07-14T19:27:05.789199+0000","last_clean":"2023-07-14T19:27:05.789199+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:27:05.789199+0000","last_undegraded":"2023-07-14T19:27:05.789199+0000","last_fullsized":"2023-07-14T19:27:05.789199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:38:02.561344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795107+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:27:05.795107+0000","last_peered":"2023-07-14T19:27:05.795107+0000","last_clean":"2023-07-14T19:27:05.795107+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:27:05.795107+0000","last_undegraded":"2023-07-14T19:27:05.795107+0000","last_fullsized":"2023-07-14T19:27:05.795107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:36:44.500967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794825+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:27:05.794825+0000","last_peered":"2023-07-14T19:27:05.794825+0000","last_clean":"2023-07-14T19:27:05.794825+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:27:05.794825+0000","last_undegraded":"2023-07-14T19:27:05.794825+0000","last_fullsized":"2023-07-14T19:27:05.794825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.697906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952069+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:27:05.952069+0000","last_peered":"2023-07-14T19:27:05.952069+0000","last_clean":"2023-07-14T19:27:05.952069+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:27:05.952069+0000","last_undegraded":"2023-07-14T19:27:05.952069+0000","last_fullsized":"2023-07-14T19:27:05.952069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787875+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:27:05.787875+0000","last_peered":"2023-07-14T19:27:05.787875+0000","last_clean":"2023-07-14T19:27:05.787875+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:27:05.787875+0000","last_undegraded":"2023-07-14T19:27:05.787875+0000","last_fullsized":"2023-07-14T19:27:05.787875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:11.270082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904385+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:27:05.904385+0000","last_peered":"2023-07-14T19:27:05.904385+0000","last_clean":"2023-07-14T19:27:05.904385+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:27:05.904385+0000","last_undegraded":"2023-07-14T19:27:05.904385+0000","last_fullsized":"2023-07-14T19:27:05.904385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:57:14.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795056+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:27:05.795056+0000","last_peered":"2023-07-14T19:27:05.795056+0000","last_clean":"2023-07-14T19:27:05.795056+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:27:05.795056+0000","last_undegraded":"2023-07-14T19:27:05.795056+0000","last_fullsized":"2023-07-14T19:27:05.795056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:09.380546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952382+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:27:05.952382+0000","last_peered":"2023-07-14T19:27:05.952382+0000","last_clean":"2023-07-14T19:27:05.952382+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:27:05.952382+0000","last_undegraded":"2023-07-14T19:27:05.952382+0000","last_fullsized":"2023-07-14T19:27:05.952382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:24.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.797079+0000","last_change":"2023-07-14T19:27:05.797079+0000","last_active":"2023-07-14T19:27:05.797079+0000","last_peered":"2023-07-14T19:27:05.797079+0000","last_clean":"2023-07-14T19:27:05.797079+0000","last_became_active":"2023-07-14T19:27:05.796765+0000","last_became_peered":"2023-07-14T19:27:05.796765+0000","last_unstale":"2023-07-14T19:27:05.797079+0000","last_undegraded":"2023-07-14T19:27:05.797079+0000","last_fullsized":"2023-07-14T19:27:05.797079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:18.699215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951759+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:27:05.951759+0000","last_peered":"2023-07-14T19:27:05.951759+0000","last_clean":"2023-07-14T19:27:05.951759+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:27:05.951759+0000","last_undegraded":"2023-07-14T19:27:05.951759+0000","last_fullsized":"2023-07-14T19:27:05.951759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:24.688894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952110+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:27:05.952110+0000","last_peered":"2023-07-14T19:27:05.952110+0000","last_clean":"2023-07-14T19:27:05.952110+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:27:05.952110+0000","last_undegraded":"2023-07-14T19:27:05.952110+0000","last_fullsized":"2023-07-14T19:27:05.952110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:11:02.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952062+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:27:05.952062+0000","last_peered":"2023-07-14T19:27:05.952062+0000","last_clean":"2023-07-14T19:27:05.952062+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:27:05.952062+0000","last_undegraded":"2023-07-14T19:27:05.952062+0000","last_fullsized":"2023-07-14T19:27:05.952062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:29.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903737+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:27:05.903737+0000","last_peered":"2023-07-14T19:27:05.903737+0000","last_clean":"2023-07-14T19:27:05.903737+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:27:05.903737+0000","last_undegraded":"2023-07-14T19:27:05.903737+0000","last_fullsized":"2023-07-14T19:27:05.903737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:14.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792938+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:27:05.792938+0000","last_peered":"2023-07-14T19:27:05.792938+0000","last_clean":"2023-07-14T19:27:05.792938+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:27:05.792938+0000","last_undegraded":"2023-07-14T19:27:05.792938+0000","last_fullsized":"2023-07-14T19:27:05.792938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:47.623791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789987+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:27:05.789987+0000","last_peered":"2023-07-14T19:27:05.789987+0000","last_clean":"2023-07-14T19:27:05.789987+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:27:05.789987+0000","last_undegraded":"2023-07-14T19:27:05.789987+0000","last_fullsized":"2023-07-14T19:27:05.789987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:45.291302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794826+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:27:05.794826+0000","last_peered":"2023-07-14T19:27:05.794826+0000","last_clean":"2023-07-14T19:27:05.794826+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:27:05.794826+0000","last_undegraded":"2023-07-14T19:27:05.794826+0000","last_fullsized":"2023-07-14T19:27:05.794826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:34:18.122288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789986+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:27:05.789986+0000","last_peered":"2023-07-14T19:27:05.789986+0000","last_clean":"2023-07-14T19:27:05.789986+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:27:05.789986+0000","last_undegraded":"2023-07-14T19:27:05.789986+0000","last_fullsized":"2023-07-14T19:27:05.789986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.693215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788293+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:27:05.788293+0000","last_peered":"2023-07-14T19:27:05.788293+0000","last_clean":"2023-07-14T19:27:05.788293+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:27:05.788293+0000","last_undegraded":"2023-07-14T19:27:05.788293+0000","last_fullsized":"2023-07-14T19:27:05.788293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:10.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788456+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:27:05.788456+0000","last_peered":"2023-07-14T19:27:05.788456+0000","last_clean":"2023-07-14T19:27:05.788456+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:27:05.788456+0000","last_undegraded":"2023-07-14T19:27:05.788456+0000","last_fullsized":"2023-07-14T19:27:05.788456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:33.601716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952516+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:27:05.952516+0000","last_peered":"2023-07-14T19:27:05.952516+0000","last_clean":"2023-07-14T19:27:05.952516+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:27:05.952516+0000","last_undegraded":"2023-07-14T19:27:05.952516+0000","last_fullsized":"2023-07-14T19:27:05.952516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:47:04.807902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795938+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:05.795938+0000","last_peered":"2023-07-14T19:27:05.795938+0000","last_clean":"2023-07-14T19:27:05.795938+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:05.795938+0000","last_undegraded":"2023-07-14T19:27:05.795938+0000","last_fullsized":"2023-07-14T19:27:05.795938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:11.673840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903827+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:27:05.903827+0000","last_peered":"2023-07-14T19:27:05.903827+0000","last_clean":"2023-07-14T19:27:05.903827+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:27:05.903827+0000","last_undegraded":"2023-07-14T19:27:05.903827+0000","last_fullsized":"2023-07-14T19:27:05.903827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:56.838822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795738+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:27:05.795738+0000","last_peered":"2023-07-14T19:27:05.795738+0000","last_clean":"2023-07-14T19:27:05.795738+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:27:05.795738+0000","last_undegraded":"2023-07-14T19:27:05.795738+0000","last_fullsized":"2023-07-14T19:27:05.795738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:12:12.590934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952436+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:27:05.952436+0000","last_peered":"2023-07-14T19:27:05.952436+0000","last_clean":"2023-07-14T19:27:05.952436+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:27:05.952436+0000","last_undegraded":"2023-07-14T19:27:05.952436+0000","last_fullsized":"2023-07-14T19:27:05.952436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:16.730207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:39.588649+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:27:39.588649+0000","last_peered":"2023-07-14T19:27:39.588649+0000","last_clean":"2023-07-14T19:27:39.588649+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:27:39.588649+0000","last_undegraded":"2023-07-14T19:27:39.588649+0000","last_fullsized":"2023-07-14T19:27:39.588649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:07.160367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795175+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:27:05.795175+0000","last_peered":"2023-07-14T19:27:05.795175+0000","last_clean":"2023-07-14T19:27:05.795175+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:27:05.795175+0000","last_undegraded":"2023-07-14T19:27:05.795175+0000","last_fullsized":"2023-07-14T19:27:05.795175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:02.082831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794601+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:27:05.794601+0000","last_peered":"2023-07-14T19:27:05.794601+0000","last_clean":"2023-07-14T19:27:05.794601+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:27:05.794601+0000","last_undegraded":"2023-07-14T19:27:05.794601+0000","last_fullsized":"2023-07-14T19:27:05.794601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:58.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952219+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:27:05.952219+0000","last_peered":"2023-07-14T19:27:05.952219+0000","last_clean":"2023-07-14T19:27:05.952219+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:27:05.952219+0000","last_undegraded":"2023-07-14T19:27:05.952219+0000","last_fullsized":"2023-07-14T19:27:05.952219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:02.114404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951680+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:05.951680+0000","last_peered":"2023-07-14T19:27:05.951680+0000","last_clean":"2023-07-14T19:27:05.951680+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:05.951680+0000","last_undegraded":"2023-07-14T19:27:05.951680+0000","last_fullsized":"2023-07-14T19:27:05.951680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.018579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787917+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:27:05.787917+0000","last_peered":"2023-07-14T19:27:05.787917+0000","last_clean":"2023-07-14T19:27:05.787917+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:27:05.787917+0000","last_undegraded":"2023-07-14T19:27:05.787917+0000","last_fullsized":"2023-07-14T19:27:05.787917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:26.867144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951987+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:27:05.951987+0000","last_peered":"2023-07-14T19:27:05.951987+0000","last_clean":"2023-07-14T19:27:05.951987+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:27:05.951987+0000","last_undegraded":"2023-07-14T19:27:05.951987+0000","last_fullsized":"2023-07-14T19:27:05.951987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.740777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789813+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:27:05.789813+0000","last_peered":"2023-07-14T19:27:05.789813+0000","last_clean":"2023-07-14T19:27:05.789813+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:27:05.789813+0000","last_undegraded":"2023-07-14T19:27:05.789813+0000","last_fullsized":"2023-07-14T19:27:05.789813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:42.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:39.587875+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:27:39.587875+0000","last_peered":"2023-07-14T19:27:39.587875+0000","last_clean":"2023-07-14T19:27:39.587875+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:27:39.587875+0000","last_undegraded":"2023-07-14T19:27:39.587875+0000","last_fullsized":"2023-07-14T19:27:39.587875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:38.911754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904223+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:27:05.904223+0000","last_peered":"2023-07-14T19:27:05.904223+0000","last_clean":"2023-07-14T19:27:05.904223+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:27:05.904223+0000","last_undegraded":"2023-07-14T19:27:05.904223+0000","last_fullsized":"2023-07-14T19:27:05.904223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:31.761093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952474+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:05.952474+0000","last_peered":"2023-07-14T19:27:05.952474+0000","last_clean":"2023-07-14T19:27:05.952474+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:05.952474+0000","last_undegraded":"2023-07-14T19:27:05.952474+0000","last_fullsized":"2023-07-14T19:27:05.952474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:41:09.562934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952454+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:27:05.952454+0000","last_peered":"2023-07-14T19:27:05.952454+0000","last_clean":"2023-07-14T19:27:05.952454+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:27:05.952454+0000","last_undegraded":"2023-07-14T19:27:05.952454+0000","last_fullsized":"2023-07-14T19:27:05.952454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:37:51.070907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952595+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:27:05.952595+0000","last_peered":"2023-07-14T19:27:05.952595+0000","last_clean":"2023-07-14T19:27:05.952595+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:27:05.952595+0000","last_undegraded":"2023-07-14T19:27:05.952595+0000","last_fullsized":"2023-07-14T19:27:05.952595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:10.381760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904155+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:27:05.904155+0000","last_peered":"2023-07-14T19:27:05.904155+0000","last_clean":"2023-07-14T19:27:05.904155+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:27:05.904155+0000","last_undegraded":"2023-07-14T19:27:05.904155+0000","last_fullsized":"2023-07-14T19:27:05.904155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:10.272127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'25","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:40.595636+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:27:40.595636+0000","last_peered":"2023-07-14T19:27:40.595636+0000","last_clean":"2023-07-14T19:27:40.595636+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:27:40.595636+0000","last_undegraded":"2023-07-14T19:27:40.595636+0000","last_fullsized":"2023-07-14T19:27:40.595636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:43.292916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56322,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":85,"num_read_kb":96,"num_write":26,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795998+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:27:05.795998+0000","last_peered":"2023-07-14T19:27:05.795998+0000","last_clean":"2023-07-14T19:27:05.795998+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:27:05.795998+0000","last_undegraded":"2023-07-14T19:27:05.795998+0000","last_fullsized":"2023-07-14T19:27:05.795998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:55.206353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.804102+0000","last_change":"2023-07-14T19:27:05.803684+0000","last_active":"2023-07-14T19:27:05.804102+0000","last_peered":"2023-07-14T19:27:05.804102+0000","last_clean":"2023-07-14T19:27:05.804102+0000","last_became_active":"2023-07-14T19:27:05.796450+0000","last_became_peered":"2023-07-14T19:27:05.796450+0000","last_unstale":"2023-07-14T19:27:05.804102+0000","last_undegraded":"2023-07-14T19:27:05.804102+0000","last_fullsized":"2023-07-14T19:27:05.804102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:01.367033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951874+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:27:05.951874+0000","last_peered":"2023-07-14T19:27:05.951874+0000","last_clean":"2023-07-14T19:27:05.951874+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:27:05.951874+0000","last_undegraded":"2023-07-14T19:27:05.951874+0000","last_fullsized":"2023-07-14T19:27:05.951874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:45:05.169932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952203+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:05.952203+0000","last_peered":"2023-07-14T19:27:05.952203+0000","last_clean":"2023-07-14T19:27:05.952203+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:05.952203+0000","last_undegraded":"2023-07-14T19:27:05.952203+0000","last_fullsized":"2023-07-14T19:27:05.952203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:08:51.150447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952514+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:27:05.952514+0000","last_peered":"2023-07-14T19:27:05.952514+0000","last_clean":"2023-07-14T19:27:05.952514+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:27:05.952514+0000","last_undegraded":"2023-07-14T19:27:05.952514+0000","last_fullsized":"2023-07-14T19:27:05.952514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:01:15.875961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:39.594561+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:27:39.594561+0000","last_peered":"2023-07-14T19:27:39.594561+0000","last_clean":"2023-07-14T19:27:39.594561+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:27:39.594561+0000","last_undegraded":"2023-07-14T19:27:39.594561+0000","last_fullsized":"2023-07-14T19:27:39.594561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:27.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952015+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:27:05.952015+0000","last_peered":"2023-07-14T19:27:05.952015+0000","last_clean":"2023-07-14T19:27:05.952015+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:27:05.952015+0000","last_undegraded":"2023-07-14T19:27:05.952015+0000","last_fullsized":"2023-07-14T19:27:05.952015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:10.265120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795312+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:05.795312+0000","last_peered":"2023-07-14T19:27:05.795312+0000","last_clean":"2023-07-14T19:27:05.795312+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:05.795312+0000","last_undegraded":"2023-07-14T19:27:05.795312+0000","last_fullsized":"2023-07-14T19:27:05.795312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:17:12.935696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.906040+0000","last_change":"2023-07-14T19:27:05.906040+0000","last_active":"2023-07-14T19:27:05.906040+0000","last_peered":"2023-07-14T19:27:05.906040+0000","last_clean":"2023-07-14T19:27:05.906040+0000","last_became_active":"2023-07-14T19:27:05.905661+0000","last_became_peered":"2023-07-14T19:27:05.905661+0000","last_unstale":"2023-07-14T19:27:05.906040+0000","last_undegraded":"2023-07-14T19:27:05.906040+0000","last_fullsized":"2023-07-14T19:27:05.906040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:29.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795885+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:27:05.795885+0000","last_peered":"2023-07-14T19:27:05.795885+0000","last_clean":"2023-07-14T19:27:05.795885+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:27:05.795885+0000","last_undegraded":"2023-07-14T19:27:05.795885+0000","last_fullsized":"2023-07-14T19:27:05.795885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:46.077263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792379+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:27:05.792379+0000","last_peered":"2023-07-14T19:27:05.792379+0000","last_clean":"2023-07-14T19:27:05.792379+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:27:05.792379+0000","last_undegraded":"2023-07-14T19:27:05.792379+0000","last_fullsized":"2023-07-14T19:27:05.792379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:30.860044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.790247+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:27:05.790247+0000","last_peered":"2023-07-14T19:27:05.790247+0000","last_clean":"2023-07-14T19:27:05.790247+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:27:05.790247+0000","last_undegraded":"2023-07-14T19:27:05.790247+0000","last_fullsized":"2023-07-14T19:27:05.790247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:44:21.663771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794852+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:27:05.794852+0000","last_peered":"2023-07-14T19:27:05.794852+0000","last_clean":"2023-07-14T19:27:05.794852+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:27:05.794852+0000","last_undegraded":"2023-07-14T19:27:05.794852+0000","last_fullsized":"2023-07-14T19:27:05.794852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:38:26.970479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952264+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:27:05.952264+0000","last_peered":"2023-07-14T19:27:05.952264+0000","last_clean":"2023-07-14T19:27:05.952264+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:27:05.952264+0000","last_undegraded":"2023-07-14T19:27:05.952264+0000","last_fullsized":"2023-07-14T19:27:05.952264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:28.081451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794859+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:27:05.794859+0000","last_peered":"2023-07-14T19:27:05.794859+0000","last_clean":"2023-07-14T19:27:05.794859+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:27:05.794859+0000","last_undegraded":"2023-07-14T19:27:05.794859+0000","last_fullsized":"2023-07-14T19:27:05.794859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:40.592640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952149+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:27:05.952149+0000","last_peered":"2023-07-14T19:27:05.952149+0000","last_clean":"2023-07-14T19:27:05.952149+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:27:05.952149+0000","last_undegraded":"2023-07-14T19:27:05.952149+0000","last_fullsized":"2023-07-14T19:27:05.952149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:29.089446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:40.595479+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:27:40.595479+0000","last_peered":"2023-07-14T19:27:40.595479+0000","last_clean":"2023-07-14T19:27:40.595479+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:27:40.595479+0000","last_undegraded":"2023-07-14T19:27:40.595479+0000","last_fullsized":"2023-07-14T19:27:40.595479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:52:38.820059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788147+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:27:05.788147+0000","last_peered":"2023-07-14T19:27:05.788147+0000","last_clean":"2023-07-14T19:27:05.788147+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:27:05.788147+0000","last_undegraded":"2023-07-14T19:27:05.788147+0000","last_fullsized":"2023-07-14T19:27:05.788147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:20.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789079+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:27:05.789079+0000","last_peered":"2023-07-14T19:27:05.789079+0000","last_clean":"2023-07-14T19:27:05.789079+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:27:05.789079+0000","last_undegraded":"2023-07-14T19:27:05.789079+0000","last_fullsized":"2023-07-14T19:27:05.789079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:11.895409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952086+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:27:05.952086+0000","last_peered":"2023-07-14T19:27:05.952086+0000","last_clean":"2023-07-14T19:27:05.952086+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:27:05.952086+0000","last_undegraded":"2023-07-14T19:27:05.952086+0000","last_fullsized":"2023-07-14T19:27:05.952086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:57.503183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951549+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:27:05.951549+0000","last_peered":"2023-07-14T19:27:05.951549+0000","last_clean":"2023-07-14T19:27:05.951549+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:27:05.951549+0000","last_undegraded":"2023-07-14T19:27:05.951549+0000","last_fullsized":"2023-07-14T19:27:05.951549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:37.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952256+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:05.952256+0000","last_peered":"2023-07-14T19:27:05.952256+0000","last_clean":"2023-07-14T19:27:05.952256+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:05.952256+0000","last_undegraded":"2023-07-14T19:27:05.952256+0000","last_fullsized":"2023-07-14T19:27:05.952256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.155552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795661+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:05.795661+0000","last_peered":"2023-07-14T19:27:05.795661+0000","last_clean":"2023-07-14T19:27:05.795661+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:05.795661+0000","last_undegraded":"2023-07-14T19:27:05.795661+0000","last_fullsized":"2023-07-14T19:27:05.795661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:05.271093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951822+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:27:05.951822+0000","last_peered":"2023-07-14T19:27:05.951822+0000","last_clean":"2023-07-14T19:27:05.951822+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:27:05.951822+0000","last_undegraded":"2023-07-14T19:27:05.951822+0000","last_fullsized":"2023-07-14T19:27:05.951822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:12:18.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903482+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:27:05.903482+0000","last_peered":"2023-07-14T19:27:05.903482+0000","last_clean":"2023-07-14T19:27:05.903482+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:27:05.903482+0000","last_undegraded":"2023-07-14T19:27:05.903482+0000","last_fullsized":"2023-07-14T19:27:05.903482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:26.769682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788286+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:27:05.788286+0000","last_peered":"2023-07-14T19:27:05.788286+0000","last_clean":"2023-07-14T19:27:05.788286+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:27:05.788286+0000","last_undegraded":"2023-07-14T19:27:05.788286+0000","last_fullsized":"2023-07-14T19:27:05.788286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:14.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788996+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:27:05.788996+0000","last_peered":"2023-07-14T19:27:05.788996+0000","last_clean":"2023-07-14T19:27:05.788996+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:27:05.788996+0000","last_undegraded":"2023-07-14T19:27:05.788996+0000","last_fullsized":"2023-07-14T19:27:05.788996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:04:44.601003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952409+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:27:05.952409+0000","last_peered":"2023-07-14T19:27:05.952409+0000","last_clean":"2023-07-14T19:27:05.952409+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:27:05.952409+0000","last_undegraded":"2023-07-14T19:27:05.952409+0000","last_fullsized":"2023-07-14T19:27:05.952409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:45:38.640268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903756+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:05.903756+0000","last_peered":"2023-07-14T19:27:05.903756+0000","last_clean":"2023-07-14T19:27:05.903756+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:05.903756+0000","last_undegraded":"2023-07-14T19:27:05.903756+0000","last_fullsized":"2023-07-14T19:27:05.903756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:31.831826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795108+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:27:05.795108+0000","last_peered":"2023-07-14T19:27:05.795108+0000","last_clean":"2023-07-14T19:27:05.795108+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:27:05.795108+0000","last_undegraded":"2023-07-14T19:27:05.795108+0000","last_fullsized":"2023-07-14T19:27:05.795108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:50.824483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787960+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:05.787960+0000","last_peered":"2023-07-14T19:27:05.787960+0000","last_clean":"2023-07-14T19:27:05.787960+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:05.787960+0000","last_undegraded":"2023-07-14T19:27:05.787960+0000","last_fullsized":"2023-07-14T19:27:05.787960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:47.286623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792034+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:27:05.792034+0000","last_peered":"2023-07-14T19:27:05.792034+0000","last_clean":"2023-07-14T19:27:05.792034+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:27:05.792034+0000","last_undegraded":"2023-07-14T19:27:05.792034+0000","last_fullsized":"2023-07-14T19:27:05.792034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:20.246759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.806328+0000","last_change":"2023-07-14T19:27:05.805846+0000","last_active":"2023-07-14T19:27:05.806328+0000","last_peered":"2023-07-14T19:27:05.806328+0000","last_clean":"2023-07-14T19:27:05.806328+0000","last_became_active":"2023-07-14T19:27:05.796805+0000","last_became_peered":"2023-07-14T19:27:05.796805+0000","last_unstale":"2023-07-14T19:27:05.806328+0000","last_undegraded":"2023-07-14T19:27:05.806328+0000","last_fullsized":"2023-07-14T19:27:05.806328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:47:58.264439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788040+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:27:05.788040+0000","last_peered":"2023-07-14T19:27:05.788040+0000","last_clean":"2023-07-14T19:27:05.788040+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:27:05.788040+0000","last_undegraded":"2023-07-14T19:27:05.788040+0000","last_fullsized":"2023-07-14T19:27:05.788040+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:54:50.760237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795490+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:05.795490+0000","last_peered":"2023-07-14T19:27:05.795490+0000","last_clean":"2023-07-14T19:27:05.795490+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:05.795490+0000","last_undegraded":"2023-07-14T19:27:05.795490+0000","last_fullsized":"2023-07-14T19:27:05.795490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:10:28.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952395+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:27:05.952395+0000","last_peered":"2023-07-14T19:27:05.952395+0000","last_clean":"2023-07-14T19:27:05.952395+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:27:05.952395+0000","last_undegraded":"2023-07-14T19:27:05.952395+0000","last_fullsized":"2023-07-14T19:27:05.952395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:47:05.779967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795623+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:05.795623+0000","last_peered":"2023-07-14T19:27:05.795623+0000","last_clean":"2023-07-14T19:27:05.795623+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:05.795623+0000","last_undegraded":"2023-07-14T19:27:05.795623+0000","last_fullsized":"2023-07-14T19:27:05.795623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:42.189785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952597+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:27:05.952597+0000","last_peered":"2023-07-14T19:27:05.952597+0000","last_clean":"2023-07-14T19:27:05.952597+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:27:05.952597+0000","last_undegraded":"2023-07-14T19:27:05.952597+0000","last_fullsized":"2023-07-14T19:27:05.952597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:31:52.615160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952458+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:27:05.952458+0000","last_peered":"2023-07-14T19:27:05.952458+0000","last_clean":"2023-07-14T19:27:05.952458+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:27:05.952458+0000","last_undegraded":"2023-07-14T19:27:05.952458+0000","last_fullsized":"2023-07-14T19:27:05.952458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:34.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952304+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:27:05.952304+0000","last_peered":"2023-07-14T19:27:05.952304+0000","last_clean":"2023-07-14T19:27:05.952304+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:27:05.952304+0000","last_undegraded":"2023-07-14T19:27:05.952304+0000","last_fullsized":"2023-07-14T19:27:05.952304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:50:19.375696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951776+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:27:05.951776+0000","last_peered":"2023-07-14T19:27:05.951776+0000","last_clean":"2023-07-14T19:27:05.951776+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:27:05.951776+0000","last_undegraded":"2023-07-14T19:27:05.951776+0000","last_fullsized":"2023-07-14T19:27:05.951776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:55.101835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952526+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:27:05.952526+0000","last_peered":"2023-07-14T19:27:05.952526+0000","last_clean":"2023-07-14T19:27:05.952526+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:27:05.952526+0000","last_undegraded":"2023-07-14T19:27:05.952526+0000","last_fullsized":"2023-07-14T19:27:05.952526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:56:02.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792195+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:27:05.792195+0000","last_peered":"2023-07-14T19:27:05.792195+0000","last_clean":"2023-07-14T19:27:05.792195+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:27:05.792195+0000","last_undegraded":"2023-07-14T19:27:05.792195+0000","last_fullsized":"2023-07-14T19:27:05.792195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:03:59.337396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794492+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:27:05.794492+0000","last_peered":"2023-07-14T19:27:05.794492+0000","last_clean":"2023-07-14T19:27:05.794492+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:27:05.794492+0000","last_undegraded":"2023-07-14T19:27:05.794492+0000","last_fullsized":"2023-07-14T19:27:05.794492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:26:45.117785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794284+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:27:05.794284+0000","last_peered":"2023-07-14T19:27:05.794284+0000","last_clean":"2023-07-14T19:27:05.794284+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:27:05.794284+0000","last_undegraded":"2023-07-14T19:27:05.794284+0000","last_fullsized":"2023-07-14T19:27:05.794284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:59.431251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952200+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:27:05.952200+0000","last_peered":"2023-07-14T19:27:05.952200+0000","last_clean":"2023-07-14T19:27:05.952200+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:27:05.952200+0000","last_undegraded":"2023-07-14T19:27:05.952200+0000","last_fullsized":"2023-07-14T19:27:05.952200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.135050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951914+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:27:05.951914+0000","last_peered":"2023-07-14T19:27:05.951914+0000","last_clean":"2023-07-14T19:27:05.951914+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:27:05.951914+0000","last_undegraded":"2023-07-14T19:27:05.951914+0000","last_fullsized":"2023-07-14T19:27:05.951914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:24.075259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792457+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:27:05.792457+0000","last_peered":"2023-07-14T19:27:05.792457+0000","last_clean":"2023-07-14T19:27:05.792457+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:27:05.792457+0000","last_undegraded":"2023-07-14T19:27:05.792457+0000","last_fullsized":"2023-07-14T19:27:05.792457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:50.553918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795123+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:27:05.795123+0000","last_peered":"2023-07-14T19:27:05.795123+0000","last_clean":"2023-07-14T19:27:05.795123+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:27:05.795123+0000","last_undegraded":"2023-07-14T19:27:05.795123+0000","last_fullsized":"2023-07-14T19:27:05.795123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:43:45.456973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788294+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:27:05.788294+0000","last_peered":"2023-07-14T19:27:05.788294+0000","last_clean":"2023-07-14T19:27:05.788294+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:27:05.788294+0000","last_undegraded":"2023-07-14T19:27:05.788294+0000","last_fullsized":"2023-07-14T19:27:05.788294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:43.308371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952030+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:27:05.952030+0000","last_peered":"2023-07-14T19:27:05.952030+0000","last_clean":"2023-07-14T19:27:05.952030+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:27:05.952030+0000","last_undegraded":"2023-07-14T19:27:05.952030+0000","last_fullsized":"2023-07-14T19:27:05.952030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:02:45.526660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787917+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:27:05.787917+0000","last_peered":"2023-07-14T19:27:05.787917+0000","last_clean":"2023-07-14T19:27:05.787917+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:27:05.787917+0000","last_undegraded":"2023-07-14T19:27:05.787917+0000","last_fullsized":"2023-07-14T19:27:05.787917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:50.377751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904094+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:27:05.904094+0000","last_peered":"2023-07-14T19:27:05.904094+0000","last_clean":"2023-07-14T19:27:05.904094+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:27:05.904094+0000","last_undegraded":"2023-07-14T19:27:05.904094+0000","last_fullsized":"2023-07-14T19:27:05.904094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:51:09.749154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951816+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:27:05.951816+0000","last_peered":"2023-07-14T19:27:05.951816+0000","last_clean":"2023-07-14T19:27:05.951816+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:27:05.951816+0000","last_undegraded":"2023-07-14T19:27:05.951816+0000","last_fullsized":"2023-07-14T19:27:05.951816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:02.143813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.796530+0000","last_change":"2023-07-14T19:27:05.796530+0000","last_active":"2023-07-14T19:27:05.796530+0000","last_peered":"2023-07-14T19:27:05.796530+0000","last_clean":"2023-07-14T19:27:05.796530+0000","last_became_active":"2023-07-14T19:27:05.796148+0000","last_became_peered":"2023-07-14T19:27:05.796148+0000","last_unstale":"2023-07-14T19:27:05.796530+0000","last_undegraded":"2023-07-14T19:27:05.796530+0000","last_fullsized":"2023-07-14T19:27:05.796530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:52.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":261,"num_read_kb":194,"num_write":147,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":180678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":5.8380000000000001,"5min":5.8380000000000001,"15min":5.8380000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.74399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.83899999999999997}]},{"osd":10,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.996}]}]},{"osd":10,"up_from":69,"seq":296352743449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1079535,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.623},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.745},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84899999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88900000000000001}]},{"osd":11,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76000000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":432010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.32800000000000001}]},{"osd":1,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.434},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.374}]},{"osd":4,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.63}]},{"osd":8,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.747},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.72799999999999998}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":431930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64000000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.5,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.496}]},{"osd":3,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.751,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88700000000000001,"5min":1.534,"15min":1.534},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.628},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86599999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.675,"15min":1.675},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.75800000000000001}]},{"osd":9,"last update":"Fri Jul 14 19:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.80000000000000004}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.86399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":1023438,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.505},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.61199999999999999,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.624}]},{"osd":4,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81999999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44600000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.60999999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.751,"15min":0.71099999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.8310000000000004,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.65100000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.753},{"interface":"front","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1,"5min":1,"15min":1},"last":0.72999999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:26:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.81000000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.78500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":431405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.54200000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.307}]},{"osd":1,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":1.212,"15min":1.212},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.63800000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.432}]},{"osd":2,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56100000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75600000000000001,"5min":1.115,"15min":1.115},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.032,"15min":1.032},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.095,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.49099999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81899999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.622},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.67700000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69699999999999995,"15min":0.71999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3480000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67600000000000005,"5min":1.55,"15min":1.55},"last":0.58099999999999996}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":487514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.441,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.53500000000000003,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.872},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81599999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.746}]},{"osd":3,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84699999999999998,"5min":1.302,"15min":1.302},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45900000000000002,"15min":0.44500000000000001},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71899999999999997,"5min":1.042,"15min":1.042},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Fri Jul 14 19:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.66000000000000003,"15min":0.64300000000000002},"min":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.66000000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.93400000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.98699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":432670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70199999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69499999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60899999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.373},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.29199999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.34999999999999998}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.622,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71299999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76900000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.438},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60199999999999998}]},{"osd":11,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":489172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.42499999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.50900000000000001,"15min":0.47299999999999998},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.79400000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.441,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.441},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54700000000000004,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.255,"15min":0.255},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.375}]},{"osd":3,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60799999999999998,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.9359999999999999,"5min":4.9359999999999999,"15min":4.9359999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46600000000000003,"15min":0.443},"min":{"1min":0.32300000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.51400000000000001},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.7560000000000002,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.753}]},{"osd":9,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32100000000000001},"min":{"1min":0.28299999999999997,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55300000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35099999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.63},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68000000000000005,"5min":1.202,"15min":1.202},"last":0.68000000000000005}]},{"osd":2,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.54300000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.438,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72399999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59599999999999997}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":431788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.28899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.434}]},{"osd":2,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.85399999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.48299999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53400000000000003}]},{"osd":8,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.879},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85899999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.745}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":1023170,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.48399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.78300000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66900000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.379}]},{"osd":3,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82999999999999996,"5min":1.022,"15min":1.022},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":1.044,"15min":1.044},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.623,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83299999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86499999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81299999999999994,"5min":1.087,"15min":1.087},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":1.113,"15min":1.113},"last":0.755}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.876,"5min":1.004,"15min":1.004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66600000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91500000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60099999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.81100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:27:43.916 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph pg dump --format=json 2023-07-14T19:27:43.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:43 smithi133 ceph-mon[128894]: pgmap v272: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:27:43.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:43 smithi161 ceph-mon[129184]: pgmap v272: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:27:44.467 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:44.857 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:44 smithi082 ceph-mon[127461]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:44.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:44 smithi133 ceph-mon[128894]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:44.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:44 smithi161 ceph-mon[129184]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:45.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1587402215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a0101150 msgr2=0x7f76a010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:45.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1587402215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a0101150 0x7f76a010bb40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f769800b0d0 tx=0x7f769800b3e0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1587402215 shutdown_connections 2023-07-14T19:27:45.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1587402215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a0101150 0x7f76a010bb40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1587402215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a00fef40 0x7f76a00ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1587402215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a01007a0 0x7f76a0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1587402215 >> 172.21.15.82:0/1587402215 conn(0x7f76a0075c90 msgr2=0x7f76a0076090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:45.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1587402215 shutdown_connections 2023-07-14T19:27:45.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.684+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1587402215 wait complete. 2023-07-14T19:27:45.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 Processor -- start 2023-07-14T19:27:45.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 -- start start 2023-07-14T19:27:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 0x7f76a0196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a01007a0 0x7f76a01971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a0101150 0x7f76a01a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0197ae0 con 0x7f76a00fef40 2023-07-14T19:27:45.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01a2250 con 0x7f76a0101150 2023-07-14T19:27:45.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a6bfa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01a23c0 con 0x7f76a01007a0 2023-07-14T19:27:45.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a4996700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 0x7f76a0196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:45.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a4996700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 0x7f76a0196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41348/0 (socket says 172.21.15.82:41348) 2023-07-14T19:27:45.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.685+0000 7f76a4996700 1 -- 172.21.15.82:0/1683851759 learned_addr learned my addr 172.21.15.82:0/1683851759 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:45.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f769ffff700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a01007a0 0x7f76a01971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:45.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a5197700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a0101150 0x7f76a01a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:45.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a4996700 1 -- 172.21.15.82:0/1683851759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a01007a0 msgr2=0x7f76a01971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a4996700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a01007a0 0x7f76a01971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a4996700 1 -- 172.21.15.82:0/1683851759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a0101150 msgr2=0x7f76a01a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a4996700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a0101150 0x7f76a01a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a4996700 1 -- 172.21.15.82:0/1683851759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009d00 con 0x7f76a00fef40 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a5197700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a0101150 0x7f76a01a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a4996700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 0x7f76a0196c90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f769400d980 tx=0x7f769400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:45.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400bb60 con 0x7f76a00fef40 2023-07-14T19:27:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7694007640 con 0x7f76a00fef40 2023-07-14T19:27:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01a26a0 con 0x7f76a00fef40 2023-07-14T19:27:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694004e90 con 0x7f76a00fef40 2023-07-14T19:27:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.686+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a01a2ba0 con 0x7f76a00fef40 2023-07-14T19:27:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.687+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a004e340 con 0x7f76a00fef40 2023-07-14T19:27:45.696 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.688+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f769400bcc0 con 0x7f76a00fef40 2023-07-14T19:27:45.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.688+0000 7f769dffb700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76900a11e0 0x7f76900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:45.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.689+0000 7f769ffff700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76900a11e0 0x7f76900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:45.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.689+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f76940cc890 con 0x7f76a00fef40 2023-07-14T19:27:45.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.689+0000 7f769ffff700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76900a11e0 0x7f76900a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f768c005fd0 tx=0x7f768c00af00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:45.698 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.693+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7694095fa0 con 0x7f76a00fef40 2023-07-14T19:27:45.877 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.876+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76a00fcc00 con 0x7f76900a11e0 2023-07-14T19:27:45.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.886+0000 7f769dffb700 1 -- 172.21.15.82:0/1683851759 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339837 (secure 0 0 0) 0x7f76a00fcc00 con 0x7f76900a11e0 2023-07-14T19:27:45.891 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:45.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76900a11e0 msgr2=0x7f76900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:45.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76900a11e0 0x7f76900a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f768c005fd0 tx=0x7f768c00af00 comp rx=0 tx=0).stop 2023-07-14T19:27:45.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 msgr2=0x7f76a0196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:45.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 0x7f76a0196c90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f769400d980 tx=0x7f769400dd40 comp rx=0 tx=0).stop 2023-07-14T19:27:45.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 shutdown_connections 2023-07-14T19:27:45.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a0101150 0x7f76a01a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f76900a11e0 0x7f76900a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a00fef40 0x7f76a0196c90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 --2- 172.21.15.82:0/1683851759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a01007a0 0x7f76a01971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:45.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 >> 172.21.15.82:0/1683851759 conn(0x7f76a0075c90 msgr2=0x7f76a00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:45.895 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 shutdown_connections 2023-07-14T19:27:45.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:45.889+0000 7f76a6bfa700 1 -- 172.21.15.82:0/1683851759 wait complete. 2023-07-14T19:27:45.896 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:27:45.914 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:45 smithi082 ceph-mon[127461]: pgmap v273: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:45.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:45 smithi133 ceph-mon[128894]: pgmap v273: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:45.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:45 smithi161 ceph-mon[129184]: pgmap v273: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:46.321 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2023-07-14T19:27:44.728602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650840,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":397,"num_read_kb":313,"num_write":330,"num_write_kb":3039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335216,"kb_used_data":10972,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606608,"statfs":{"total":1151940427776,"available":1151597166592,"internally_reserved":0,"allocated":11235328,"data_stored":7126162,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003005"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792664+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:27:05.792664+0000","last_peered":"2023-07-14T19:27:05.792664+0000","last_clean":"2023-07-14T19:27:05.792664+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:27:05.792664+0000","last_undegraded":"2023-07-14T19:27:05.792664+0000","last_fullsized":"2023-07-14T19:27:05.792664+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:41.471339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952245+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:27:05.952245+0000","last_peered":"2023-07-14T19:27:05.952245+0000","last_clean":"2023-07-14T19:27:05.952245+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:27:05.952245+0000","last_undegraded":"2023-07-14T19:27:05.952245+0000","last_fullsized":"2023-07-14T19:27:05.952245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:01.099918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952318+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:27:05.952318+0000","last_peered":"2023-07-14T19:27:05.952318+0000","last_clean":"2023-07-14T19:27:05.952318+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:27:05.952318+0000","last_undegraded":"2023-07-14T19:27:05.952318+0000","last_fullsized":"2023-07-14T19:27:05.952318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:33:44.231618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789199+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:27:05.789199+0000","last_peered":"2023-07-14T19:27:05.789199+0000","last_clean":"2023-07-14T19:27:05.789199+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:27:05.789199+0000","last_undegraded":"2023-07-14T19:27:05.789199+0000","last_fullsized":"2023-07-14T19:27:05.789199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:38:02.561344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795107+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:27:05.795107+0000","last_peered":"2023-07-14T19:27:05.795107+0000","last_clean":"2023-07-14T19:27:05.795107+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:27:05.795107+0000","last_undegraded":"2023-07-14T19:27:05.795107+0000","last_fullsized":"2023-07-14T19:27:05.795107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:36:44.500967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794825+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:27:05.794825+0000","last_peered":"2023-07-14T19:27:05.794825+0000","last_clean":"2023-07-14T19:27:05.794825+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:27:05.794825+0000","last_undegraded":"2023-07-14T19:27:05.794825+0000","last_fullsized":"2023-07-14T19:27:05.794825+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.697906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952069+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:27:05.952069+0000","last_peered":"2023-07-14T19:27:05.952069+0000","last_clean":"2023-07-14T19:27:05.952069+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:27:05.952069+0000","last_undegraded":"2023-07-14T19:27:05.952069+0000","last_fullsized":"2023-07-14T19:27:05.952069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787875+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:27:05.787875+0000","last_peered":"2023-07-14T19:27:05.787875+0000","last_clean":"2023-07-14T19:27:05.787875+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:27:05.787875+0000","last_undegraded":"2023-07-14T19:27:05.787875+0000","last_fullsized":"2023-07-14T19:27:05.787875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:11.270082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904385+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:27:05.904385+0000","last_peered":"2023-07-14T19:27:05.904385+0000","last_clean":"2023-07-14T19:27:05.904385+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:27:05.904385+0000","last_undegraded":"2023-07-14T19:27:05.904385+0000","last_fullsized":"2023-07-14T19:27:05.904385+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:57:14.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795056+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:27:05.795056+0000","last_peered":"2023-07-14T19:27:05.795056+0000","last_clean":"2023-07-14T19:27:05.795056+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:27:05.795056+0000","last_undegraded":"2023-07-14T19:27:05.795056+0000","last_fullsized":"2023-07-14T19:27:05.795056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:09.380546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952382+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:27:05.952382+0000","last_peered":"2023-07-14T19:27:05.952382+0000","last_clean":"2023-07-14T19:27:05.952382+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:27:05.952382+0000","last_undegraded":"2023-07-14T19:27:05.952382+0000","last_fullsized":"2023-07-14T19:27:05.952382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:24.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.797079+0000","last_change":"2023-07-14T19:27:05.797079+0000","last_active":"2023-07-14T19:27:05.797079+0000","last_peered":"2023-07-14T19:27:05.797079+0000","last_clean":"2023-07-14T19:27:05.797079+0000","last_became_active":"2023-07-14T19:27:05.796765+0000","last_became_peered":"2023-07-14T19:27:05.796765+0000","last_unstale":"2023-07-14T19:27:05.797079+0000","last_undegraded":"2023-07-14T19:27:05.797079+0000","last_fullsized":"2023-07-14T19:27:05.797079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:18.699215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951759+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:27:05.951759+0000","last_peered":"2023-07-14T19:27:05.951759+0000","last_clean":"2023-07-14T19:27:05.951759+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:27:05.951759+0000","last_undegraded":"2023-07-14T19:27:05.951759+0000","last_fullsized":"2023-07-14T19:27:05.951759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:24.688894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952110+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:27:05.952110+0000","last_peered":"2023-07-14T19:27:05.952110+0000","last_clean":"2023-07-14T19:27:05.952110+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:27:05.952110+0000","last_undegraded":"2023-07-14T19:27:05.952110+0000","last_fullsized":"2023-07-14T19:27:05.952110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:11:02.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952062+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:27:05.952062+0000","last_peered":"2023-07-14T19:27:05.952062+0000","last_clean":"2023-07-14T19:27:05.952062+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:27:05.952062+0000","last_undegraded":"2023-07-14T19:27:05.952062+0000","last_fullsized":"2023-07-14T19:27:05.952062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:29.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903737+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:27:05.903737+0000","last_peered":"2023-07-14T19:27:05.903737+0000","last_clean":"2023-07-14T19:27:05.903737+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:27:05.903737+0000","last_undegraded":"2023-07-14T19:27:05.903737+0000","last_fullsized":"2023-07-14T19:27:05.903737+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:14.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792938+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:27:05.792938+0000","last_peered":"2023-07-14T19:27:05.792938+0000","last_clean":"2023-07-14T19:27:05.792938+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:27:05.792938+0000","last_undegraded":"2023-07-14T19:27:05.792938+0000","last_fullsized":"2023-07-14T19:27:05.792938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:47.623791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789987+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:27:05.789987+0000","last_peered":"2023-07-14T19:27:05.789987+0000","last_clean":"2023-07-14T19:27:05.789987+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:27:05.789987+0000","last_undegraded":"2023-07-14T19:27:05.789987+0000","last_fullsized":"2023-07-14T19:27:05.789987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:45.291302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794826+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:27:05.794826+0000","last_peered":"2023-07-14T19:27:05.794826+0000","last_clean":"2023-07-14T19:27:05.794826+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:27:05.794826+0000","last_undegraded":"2023-07-14T19:27:05.794826+0000","last_fullsized":"2023-07-14T19:27:05.794826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:34:18.122288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789986+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:27:05.789986+0000","last_peered":"2023-07-14T19:27:05.789986+0000","last_clean":"2023-07-14T19:27:05.789986+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:27:05.789986+0000","last_undegraded":"2023-07-14T19:27:05.789986+0000","last_fullsized":"2023-07-14T19:27:05.789986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.693215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788293+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:27:05.788293+0000","last_peered":"2023-07-14T19:27:05.788293+0000","last_clean":"2023-07-14T19:27:05.788293+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:27:05.788293+0000","last_undegraded":"2023-07-14T19:27:05.788293+0000","last_fullsized":"2023-07-14T19:27:05.788293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:10.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788456+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:27:05.788456+0000","last_peered":"2023-07-14T19:27:05.788456+0000","last_clean":"2023-07-14T19:27:05.788456+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:27:05.788456+0000","last_undegraded":"2023-07-14T19:27:05.788456+0000","last_fullsized":"2023-07-14T19:27:05.788456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:33.601716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952516+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:27:05.952516+0000","last_peered":"2023-07-14T19:27:05.952516+0000","last_clean":"2023-07-14T19:27:05.952516+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:27:05.952516+0000","last_undegraded":"2023-07-14T19:27:05.952516+0000","last_fullsized":"2023-07-14T19:27:05.952516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:47:04.807902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795938+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:05.795938+0000","last_peered":"2023-07-14T19:27:05.795938+0000","last_clean":"2023-07-14T19:27:05.795938+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:05.795938+0000","last_undegraded":"2023-07-14T19:27:05.795938+0000","last_fullsized":"2023-07-14T19:27:05.795938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:11.673840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903827+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:27:05.903827+0000","last_peered":"2023-07-14T19:27:05.903827+0000","last_clean":"2023-07-14T19:27:05.903827+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:27:05.903827+0000","last_undegraded":"2023-07-14T19:27:05.903827+0000","last_fullsized":"2023-07-14T19:27:05.903827+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:56.838822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795738+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:27:05.795738+0000","last_peered":"2023-07-14T19:27:05.795738+0000","last_clean":"2023-07-14T19:27:05.795738+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:27:05.795738+0000","last_undegraded":"2023-07-14T19:27:05.795738+0000","last_fullsized":"2023-07-14T19:27:05.795738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:12:12.590934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952436+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:27:05.952436+0000","last_peered":"2023-07-14T19:27:05.952436+0000","last_clean":"2023-07-14T19:27:05.952436+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:27:05.952436+0000","last_undegraded":"2023-07-14T19:27:05.952436+0000","last_fullsized":"2023-07-14T19:27:05.952436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:16.730207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:39.588649+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:27:39.588649+0000","last_peered":"2023-07-14T19:27:39.588649+0000","last_clean":"2023-07-14T19:27:39.588649+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:27:39.588649+0000","last_undegraded":"2023-07-14T19:27:39.588649+0000","last_fullsized":"2023-07-14T19:27:39.588649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:07.160367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795175+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:27:05.795175+0000","last_peered":"2023-07-14T19:27:05.795175+0000","last_clean":"2023-07-14T19:27:05.795175+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:27:05.795175+0000","last_undegraded":"2023-07-14T19:27:05.795175+0000","last_fullsized":"2023-07-14T19:27:05.795175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:02.082831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794601+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:27:05.794601+0000","last_peered":"2023-07-14T19:27:05.794601+0000","last_clean":"2023-07-14T19:27:05.794601+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:27:05.794601+0000","last_undegraded":"2023-07-14T19:27:05.794601+0000","last_fullsized":"2023-07-14T19:27:05.794601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:58.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952219+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:27:05.952219+0000","last_peered":"2023-07-14T19:27:05.952219+0000","last_clean":"2023-07-14T19:27:05.952219+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:27:05.952219+0000","last_undegraded":"2023-07-14T19:27:05.952219+0000","last_fullsized":"2023-07-14T19:27:05.952219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:02.114404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951680+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:05.951680+0000","last_peered":"2023-07-14T19:27:05.951680+0000","last_clean":"2023-07-14T19:27:05.951680+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:05.951680+0000","last_undegraded":"2023-07-14T19:27:05.951680+0000","last_fullsized":"2023-07-14T19:27:05.951680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.018579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787917+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:27:05.787917+0000","last_peered":"2023-07-14T19:27:05.787917+0000","last_clean":"2023-07-14T19:27:05.787917+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:27:05.787917+0000","last_undegraded":"2023-07-14T19:27:05.787917+0000","last_fullsized":"2023-07-14T19:27:05.787917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:26.867144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951987+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:27:05.951987+0000","last_peered":"2023-07-14T19:27:05.951987+0000","last_clean":"2023-07-14T19:27:05.951987+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:27:05.951987+0000","last_undegraded":"2023-07-14T19:27:05.951987+0000","last_fullsized":"2023-07-14T19:27:05.951987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.740777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789813+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:27:05.789813+0000","last_peered":"2023-07-14T19:27:05.789813+0000","last_clean":"2023-07-14T19:27:05.789813+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:27:05.789813+0000","last_undegraded":"2023-07-14T19:27:05.789813+0000","last_fullsized":"2023-07-14T19:27:05.789813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:42.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:39.587875+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:27:39.587875+0000","last_peered":"2023-07-14T19:27:39.587875+0000","last_clean":"2023-07-14T19:27:39.587875+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:27:39.587875+0000","last_undegraded":"2023-07-14T19:27:39.587875+0000","last_fullsized":"2023-07-14T19:27:39.587875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:38.911754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904223+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:27:05.904223+0000","last_peered":"2023-07-14T19:27:05.904223+0000","last_clean":"2023-07-14T19:27:05.904223+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:27:05.904223+0000","last_undegraded":"2023-07-14T19:27:05.904223+0000","last_fullsized":"2023-07-14T19:27:05.904223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:31.761093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952474+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:05.952474+0000","last_peered":"2023-07-14T19:27:05.952474+0000","last_clean":"2023-07-14T19:27:05.952474+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:05.952474+0000","last_undegraded":"2023-07-14T19:27:05.952474+0000","last_fullsized":"2023-07-14T19:27:05.952474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:41:09.562934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952454+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:27:05.952454+0000","last_peered":"2023-07-14T19:27:05.952454+0000","last_clean":"2023-07-14T19:27:05.952454+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:27:05.952454+0000","last_undegraded":"2023-07-14T19:27:05.952454+0000","last_fullsized":"2023-07-14T19:27:05.952454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:37:51.070907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952595+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:27:05.952595+0000","last_peered":"2023-07-14T19:27:05.952595+0000","last_clean":"2023-07-14T19:27:05.952595+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:27:05.952595+0000","last_undegraded":"2023-07-14T19:27:05.952595+0000","last_fullsized":"2023-07-14T19:27:05.952595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:10.381760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904155+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:27:05.904155+0000","last_peered":"2023-07-14T19:27:05.904155+0000","last_clean":"2023-07-14T19:27:05.904155+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:27:05.904155+0000","last_undegraded":"2023-07-14T19:27:05.904155+0000","last_fullsized":"2023-07-14T19:27:05.904155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:10.272127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'25","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:40.595636+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:27:40.595636+0000","last_peered":"2023-07-14T19:27:40.595636+0000","last_clean":"2023-07-14T19:27:40.595636+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:27:40.595636+0000","last_undegraded":"2023-07-14T19:27:40.595636+0000","last_fullsized":"2023-07-14T19:27:40.595636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:43.292916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56322,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":85,"num_read_kb":96,"num_write":26,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795998+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:27:05.795998+0000","last_peered":"2023-07-14T19:27:05.795998+0000","last_clean":"2023-07-14T19:27:05.795998+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:27:05.795998+0000","last_undegraded":"2023-07-14T19:27:05.795998+0000","last_fullsized":"2023-07-14T19:27:05.795998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:55.206353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.804102+0000","last_change":"2023-07-14T19:27:05.803684+0000","last_active":"2023-07-14T19:27:05.804102+0000","last_peered":"2023-07-14T19:27:05.804102+0000","last_clean":"2023-07-14T19:27:05.804102+0000","last_became_active":"2023-07-14T19:27:05.796450+0000","last_became_peered":"2023-07-14T19:27:05.796450+0000","last_unstale":"2023-07-14T19:27:05.804102+0000","last_undegraded":"2023-07-14T19:27:05.804102+0000","last_fullsized":"2023-07-14T19:27:05.804102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:01.367033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951874+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:27:05.951874+0000","last_peered":"2023-07-14T19:27:05.951874+0000","last_clean":"2023-07-14T19:27:05.951874+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:27:05.951874+0000","last_undegraded":"2023-07-14T19:27:05.951874+0000","last_fullsized":"2023-07-14T19:27:05.951874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:45:05.169932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952203+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:05.952203+0000","last_peered":"2023-07-14T19:27:05.952203+0000","last_clean":"2023-07-14T19:27:05.952203+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:05.952203+0000","last_undegraded":"2023-07-14T19:27:05.952203+0000","last_fullsized":"2023-07-14T19:27:05.952203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:08:51.150447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952514+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:27:05.952514+0000","last_peered":"2023-07-14T19:27:05.952514+0000","last_clean":"2023-07-14T19:27:05.952514+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:27:05.952514+0000","last_undegraded":"2023-07-14T19:27:05.952514+0000","last_fullsized":"2023-07-14T19:27:05.952514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:01:15.875961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'3","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:39.594561+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:27:39.594561+0000","last_peered":"2023-07-14T19:27:39.594561+0000","last_clean":"2023-07-14T19:27:39.594561+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:27:39.594561+0000","last_undegraded":"2023-07-14T19:27:39.594561+0000","last_fullsized":"2023-07-14T19:27:39.594561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:27.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952015+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:27:05.952015+0000","last_peered":"2023-07-14T19:27:05.952015+0000","last_clean":"2023-07-14T19:27:05.952015+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:27:05.952015+0000","last_undegraded":"2023-07-14T19:27:05.952015+0000","last_fullsized":"2023-07-14T19:27:05.952015+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:10.265120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795312+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:05.795312+0000","last_peered":"2023-07-14T19:27:05.795312+0000","last_clean":"2023-07-14T19:27:05.795312+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:05.795312+0000","last_undegraded":"2023-07-14T19:27:05.795312+0000","last_fullsized":"2023-07-14T19:27:05.795312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:17:12.935696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.906040+0000","last_change":"2023-07-14T19:27:05.906040+0000","last_active":"2023-07-14T19:27:05.906040+0000","last_peered":"2023-07-14T19:27:05.906040+0000","last_clean":"2023-07-14T19:27:05.906040+0000","last_became_active":"2023-07-14T19:27:05.905661+0000","last_became_peered":"2023-07-14T19:27:05.905661+0000","last_unstale":"2023-07-14T19:27:05.906040+0000","last_undegraded":"2023-07-14T19:27:05.906040+0000","last_fullsized":"2023-07-14T19:27:05.906040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:29.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795885+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:27:05.795885+0000","last_peered":"2023-07-14T19:27:05.795885+0000","last_clean":"2023-07-14T19:27:05.795885+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:27:05.795885+0000","last_undegraded":"2023-07-14T19:27:05.795885+0000","last_fullsized":"2023-07-14T19:27:05.795885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:46.077263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792379+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:27:05.792379+0000","last_peered":"2023-07-14T19:27:05.792379+0000","last_clean":"2023-07-14T19:27:05.792379+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:27:05.792379+0000","last_undegraded":"2023-07-14T19:27:05.792379+0000","last_fullsized":"2023-07-14T19:27:05.792379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:30.860044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.790247+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:27:05.790247+0000","last_peered":"2023-07-14T19:27:05.790247+0000","last_clean":"2023-07-14T19:27:05.790247+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:27:05.790247+0000","last_undegraded":"2023-07-14T19:27:05.790247+0000","last_fullsized":"2023-07-14T19:27:05.790247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:44:21.663771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794852+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:27:05.794852+0000","last_peered":"2023-07-14T19:27:05.794852+0000","last_clean":"2023-07-14T19:27:05.794852+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:27:05.794852+0000","last_undegraded":"2023-07-14T19:27:05.794852+0000","last_fullsized":"2023-07-14T19:27:05.794852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:38:26.970479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952264+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:27:05.952264+0000","last_peered":"2023-07-14T19:27:05.952264+0000","last_clean":"2023-07-14T19:27:05.952264+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:27:05.952264+0000","last_undegraded":"2023-07-14T19:27:05.952264+0000","last_fullsized":"2023-07-14T19:27:05.952264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:28.081451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794859+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:27:05.794859+0000","last_peered":"2023-07-14T19:27:05.794859+0000","last_clean":"2023-07-14T19:27:05.794859+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:27:05.794859+0000","last_undegraded":"2023-07-14T19:27:05.794859+0000","last_fullsized":"2023-07-14T19:27:05.794859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:40.592640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952149+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:27:05.952149+0000","last_peered":"2023-07-14T19:27:05.952149+0000","last_clean":"2023-07-14T19:27:05.952149+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:27:05.952149+0000","last_undegraded":"2023-07-14T19:27:05.952149+0000","last_fullsized":"2023-07-14T19:27:05.952149+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:29.089446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:40.595479+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:27:40.595479+0000","last_peered":"2023-07-14T19:27:40.595479+0000","last_clean":"2023-07-14T19:27:40.595479+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:27:40.595479+0000","last_undegraded":"2023-07-14T19:27:40.595479+0000","last_fullsized":"2023-07-14T19:27:40.595479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:52:38.820059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788147+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:27:05.788147+0000","last_peered":"2023-07-14T19:27:05.788147+0000","last_clean":"2023-07-14T19:27:05.788147+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:27:05.788147+0000","last_undegraded":"2023-07-14T19:27:05.788147+0000","last_fullsized":"2023-07-14T19:27:05.788147+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:20.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789079+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:27:05.789079+0000","last_peered":"2023-07-14T19:27:05.789079+0000","last_clean":"2023-07-14T19:27:05.789079+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:27:05.789079+0000","last_undegraded":"2023-07-14T19:27:05.789079+0000","last_fullsized":"2023-07-14T19:27:05.789079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:11.895409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952086+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:27:05.952086+0000","last_peered":"2023-07-14T19:27:05.952086+0000","last_clean":"2023-07-14T19:27:05.952086+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:27:05.952086+0000","last_undegraded":"2023-07-14T19:27:05.952086+0000","last_fullsized":"2023-07-14T19:27:05.952086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:57.503183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951549+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:27:05.951549+0000","last_peered":"2023-07-14T19:27:05.951549+0000","last_clean":"2023-07-14T19:27:05.951549+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:27:05.951549+0000","last_undegraded":"2023-07-14T19:27:05.951549+0000","last_fullsized":"2023-07-14T19:27:05.951549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:37.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952256+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:05.952256+0000","last_peered":"2023-07-14T19:27:05.952256+0000","last_clean":"2023-07-14T19:27:05.952256+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:05.952256+0000","last_undegraded":"2023-07-14T19:27:05.952256+0000","last_fullsized":"2023-07-14T19:27:05.952256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.155552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795661+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:05.795661+0000","last_peered":"2023-07-14T19:27:05.795661+0000","last_clean":"2023-07-14T19:27:05.795661+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:05.795661+0000","last_undegraded":"2023-07-14T19:27:05.795661+0000","last_fullsized":"2023-07-14T19:27:05.795661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:05.271093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951822+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:27:05.951822+0000","last_peered":"2023-07-14T19:27:05.951822+0000","last_clean":"2023-07-14T19:27:05.951822+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:27:05.951822+0000","last_undegraded":"2023-07-14T19:27:05.951822+0000","last_fullsized":"2023-07-14T19:27:05.951822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:12:18.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903482+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:27:05.903482+0000","last_peered":"2023-07-14T19:27:05.903482+0000","last_clean":"2023-07-14T19:27:05.903482+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:27:05.903482+0000","last_undegraded":"2023-07-14T19:27:05.903482+0000","last_fullsized":"2023-07-14T19:27:05.903482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:26.769682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788286+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:27:05.788286+0000","last_peered":"2023-07-14T19:27:05.788286+0000","last_clean":"2023-07-14T19:27:05.788286+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:27:05.788286+0000","last_undegraded":"2023-07-14T19:27:05.788286+0000","last_fullsized":"2023-07-14T19:27:05.788286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:14.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788996+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:27:05.788996+0000","last_peered":"2023-07-14T19:27:05.788996+0000","last_clean":"2023-07-14T19:27:05.788996+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:27:05.788996+0000","last_undegraded":"2023-07-14T19:27:05.788996+0000","last_fullsized":"2023-07-14T19:27:05.788996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:04:44.601003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952409+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:27:05.952409+0000","last_peered":"2023-07-14T19:27:05.952409+0000","last_clean":"2023-07-14T19:27:05.952409+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:27:05.952409+0000","last_undegraded":"2023-07-14T19:27:05.952409+0000","last_fullsized":"2023-07-14T19:27:05.952409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:45:38.640268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903756+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:05.903756+0000","last_peered":"2023-07-14T19:27:05.903756+0000","last_clean":"2023-07-14T19:27:05.903756+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:05.903756+0000","last_undegraded":"2023-07-14T19:27:05.903756+0000","last_fullsized":"2023-07-14T19:27:05.903756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:31.831826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795108+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:27:05.795108+0000","last_peered":"2023-07-14T19:27:05.795108+0000","last_clean":"2023-07-14T19:27:05.795108+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:27:05.795108+0000","last_undegraded":"2023-07-14T19:27:05.795108+0000","last_fullsized":"2023-07-14T19:27:05.795108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:50.824483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787960+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:05.787960+0000","last_peered":"2023-07-14T19:27:05.787960+0000","last_clean":"2023-07-14T19:27:05.787960+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:05.787960+0000","last_undegraded":"2023-07-14T19:27:05.787960+0000","last_fullsized":"2023-07-14T19:27:05.787960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:47.286623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792034+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:27:05.792034+0000","last_peered":"2023-07-14T19:27:05.792034+0000","last_clean":"2023-07-14T19:27:05.792034+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:27:05.792034+0000","last_undegraded":"2023-07-14T19:27:05.792034+0000","last_fullsized":"2023-07-14T19:27:05.792034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:20.246759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.806328+0000","last_change":"2023-07-14T19:27:05.805846+0000","last_active":"2023-07-14T19:27:05.806328+0000","last_peered":"2023-07-14T19:27:05.806328+0000","last_clean":"2023-07-14T19:27:05.806328+0000","last_became_active":"2023-07-14T19:27:05.796805+0000","last_became_peered":"2023-07-14T19:27:05.796805+0000","last_unstale":"2023-07-14T19:27:05.806328+0000","last_undegraded":"2023-07-14T19:27:05.806328+0000","last_fullsized":"2023-07-14T19:27:05.806328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:47:58.264439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788040+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:27:05.788040+0000","last_peered":"2023-07-14T19:27:05.788040+0000","last_clean":"2023-07-14T19:27:05.788040+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:27:05.788040+0000","last_undegraded":"2023-07-14T19:27:05.788040+0000","last_fullsized":"2023-07-14T19:27:05.788040+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:54:50.760237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795490+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:05.795490+0000","last_peered":"2023-07-14T19:27:05.795490+0000","last_clean":"2023-07-14T19:27:05.795490+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:05.795490+0000","last_undegraded":"2023-07-14T19:27:05.795490+0000","last_fullsized":"2023-07-14T19:27:05.795490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:10:28.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952395+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:27:05.952395+0000","last_peered":"2023-07-14T19:27:05.952395+0000","last_clean":"2023-07-14T19:27:05.952395+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:27:05.952395+0000","last_undegraded":"2023-07-14T19:27:05.952395+0000","last_fullsized":"2023-07-14T19:27:05.952395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:47:05.779967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795623+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:05.795623+0000","last_peered":"2023-07-14T19:27:05.795623+0000","last_clean":"2023-07-14T19:27:05.795623+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:05.795623+0000","last_undegraded":"2023-07-14T19:27:05.795623+0000","last_fullsized":"2023-07-14T19:27:05.795623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:42.189785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952597+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:27:05.952597+0000","last_peered":"2023-07-14T19:27:05.952597+0000","last_clean":"2023-07-14T19:27:05.952597+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:27:05.952597+0000","last_undegraded":"2023-07-14T19:27:05.952597+0000","last_fullsized":"2023-07-14T19:27:05.952597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:31:52.615160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952458+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:27:05.952458+0000","last_peered":"2023-07-14T19:27:05.952458+0000","last_clean":"2023-07-14T19:27:05.952458+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:27:05.952458+0000","last_undegraded":"2023-07-14T19:27:05.952458+0000","last_fullsized":"2023-07-14T19:27:05.952458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:34.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952304+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:27:05.952304+0000","last_peered":"2023-07-14T19:27:05.952304+0000","last_clean":"2023-07-14T19:27:05.952304+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:27:05.952304+0000","last_undegraded":"2023-07-14T19:27:05.952304+0000","last_fullsized":"2023-07-14T19:27:05.952304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:50:19.375696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951776+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:27:05.951776+0000","last_peered":"2023-07-14T19:27:05.951776+0000","last_clean":"2023-07-14T19:27:05.951776+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:27:05.951776+0000","last_undegraded":"2023-07-14T19:27:05.951776+0000","last_fullsized":"2023-07-14T19:27:05.951776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:55.101835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952526+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:27:05.952526+0000","last_peered":"2023-07-14T19:27:05.952526+0000","last_clean":"2023-07-14T19:27:05.952526+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:27:05.952526+0000","last_undegraded":"2023-07-14T19:27:05.952526+0000","last_fullsized":"2023-07-14T19:27:05.952526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:56:02.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792195+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:27:05.792195+0000","last_peered":"2023-07-14T19:27:05.792195+0000","last_clean":"2023-07-14T19:27:05.792195+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:27:05.792195+0000","last_undegraded":"2023-07-14T19:27:05.792195+0000","last_fullsized":"2023-07-14T19:27:05.792195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:03:59.337396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794492+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:27:05.794492+0000","last_peered":"2023-07-14T19:27:05.794492+0000","last_clean":"2023-07-14T19:27:05.794492+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:27:05.794492+0000","last_undegraded":"2023-07-14T19:27:05.794492+0000","last_fullsized":"2023-07-14T19:27:05.794492+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:26:45.117785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794284+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:27:05.794284+0000","last_peered":"2023-07-14T19:27:05.794284+0000","last_clean":"2023-07-14T19:27:05.794284+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:27:05.794284+0000","last_undegraded":"2023-07-14T19:27:05.794284+0000","last_fullsized":"2023-07-14T19:27:05.794284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:59.431251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952200+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:27:05.952200+0000","last_peered":"2023-07-14T19:27:05.952200+0000","last_clean":"2023-07-14T19:27:05.952200+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:27:05.952200+0000","last_undegraded":"2023-07-14T19:27:05.952200+0000","last_fullsized":"2023-07-14T19:27:05.952200+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.135050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951914+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:27:05.951914+0000","last_peered":"2023-07-14T19:27:05.951914+0000","last_clean":"2023-07-14T19:27:05.951914+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:27:05.951914+0000","last_undegraded":"2023-07-14T19:27:05.951914+0000","last_fullsized":"2023-07-14T19:27:05.951914+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:24.075259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.792457+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:27:05.792457+0000","last_peered":"2023-07-14T19:27:05.792457+0000","last_clean":"2023-07-14T19:27:05.792457+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:27:05.792457+0000","last_undegraded":"2023-07-14T19:27:05.792457+0000","last_fullsized":"2023-07-14T19:27:05.792457+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:50.553918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795123+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:27:05.795123+0000","last_peered":"2023-07-14T19:27:05.795123+0000","last_clean":"2023-07-14T19:27:05.795123+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:27:05.795123+0000","last_undegraded":"2023-07-14T19:27:05.795123+0000","last_fullsized":"2023-07-14T19:27:05.795123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:43:45.456973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788294+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:27:05.788294+0000","last_peered":"2023-07-14T19:27:05.788294+0000","last_clean":"2023-07-14T19:27:05.788294+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:27:05.788294+0000","last_undegraded":"2023-07-14T19:27:05.788294+0000","last_fullsized":"2023-07-14T19:27:05.788294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:43.308371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952030+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:27:05.952030+0000","last_peered":"2023-07-14T19:27:05.952030+0000","last_clean":"2023-07-14T19:27:05.952030+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:27:05.952030+0000","last_undegraded":"2023-07-14T19:27:05.952030+0000","last_fullsized":"2023-07-14T19:27:05.952030+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:02:45.526660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787917+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:27:05.787917+0000","last_peered":"2023-07-14T19:27:05.787917+0000","last_clean":"2023-07-14T19:27:05.787917+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:27:05.787917+0000","last_undegraded":"2023-07-14T19:27:05.787917+0000","last_fullsized":"2023-07-14T19:27:05.787917+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:50.377751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.904094+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:27:05.904094+0000","last_peered":"2023-07-14T19:27:05.904094+0000","last_clean":"2023-07-14T19:27:05.904094+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:27:05.904094+0000","last_undegraded":"2023-07-14T19:27:05.904094+0000","last_fullsized":"2023-07-14T19:27:05.904094+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:51:09.749154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951816+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:27:05.951816+0000","last_peered":"2023-07-14T19:27:05.951816+0000","last_clean":"2023-07-14T19:27:05.951816+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:27:05.951816+0000","last_undegraded":"2023-07-14T19:27:05.951816+0000","last_fullsized":"2023-07-14T19:27:05.951816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:02.143813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.796530+0000","last_change":"2023-07-14T19:27:05.796530+0000","last_active":"2023-07-14T19:27:05.796530+0000","last_peered":"2023-07-14T19:27:05.796530+0000","last_clean":"2023-07-14T19:27:05.796530+0000","last_became_active":"2023-07-14T19:27:05.796148+0000","last_became_peered":"2023-07-14T19:27:05.796148+0000","last_unstale":"2023-07-14T19:27:05.796530+0000","last_undegraded":"2023-07-14T19:27:05.796530+0000","last_fullsized":"2023-07-14T19:27:05.796530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:52.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":261,"num_read_kb":194,"num_write":147,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":180678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514517,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":5.8380000000000001,"5min":5.8380000000000001,"15min":5.8380000000000001},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.74399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":9,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.0170000000000003,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.83899999999999997}]},{"osd":10,"last update":"Fri Jul 14 19:27:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.996}]}]},{"osd":10,"up_from":69,"seq":296352743449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1079535,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.623},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.745},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84899999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88900000000000001}]},{"osd":11,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76000000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":432010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.32800000000000001}]},{"osd":1,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.434},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.374}]},{"osd":4,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.63}]},{"osd":8,"last update":"Fri Jul 14 19:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.747},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.65400000000000003}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.72799999999999998}]}]},{"osd":8,"up_from":57,"seq":244813135905,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":431930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64000000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.498},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.5,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.751,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88700000000000001,"5min":1.534,"15min":1.534},"last":0.625}]},{"osd":4,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.628},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86599999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.874},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.675,"15min":1.675},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.98599999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.56699999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90300000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":1.014},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.93300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":7,"up_from":50,"seq":214748364838,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":1023438,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.434,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60999999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.374},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.502}]},{"osd":3,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.61199999999999999,"15min":0.57599999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.8019999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.624,"15min":0.60199999999999998},"min":{"1min":0.52100000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81999999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.378,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53500000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.627,"15min":0.61499999999999999},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.60999999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.751,"15min":0.71099999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":5.8310000000000004,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.66800000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1,"5min":1,"15min":1},"last":0.77900000000000003}]},{"osd":9,"last update":"Fri Jul 14 19:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.76900000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.7610000000000001,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.72799999999999998}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":6,"up_from":45,"seq":193273528361,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":897024,"data_stored":431405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.54200000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.307}]},{"osd":1,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":1.212,"15min":1.212},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.63800000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.432}]},{"osd":2,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56100000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75600000000000001,"5min":1.115,"15min":1.115},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.032,"15min":1.032},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.095,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68600000000000005,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.49099999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.72099999999999997,"15min":0.73399999999999999},"min":{"1min":0.50600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81899999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.622},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.67700000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69699999999999995,"15min":0.71999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3480000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67600000000000005,"5min":1.55,"15min":1.55},"last":0.58099999999999996}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.55600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":487514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.441,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.53500000000000003,"15min":0.49299999999999999},"min":{"1min":0.35899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.872},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.36899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81599999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.746}]},{"osd":3,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84699999999999998,"5min":1.302,"15min":1.302},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.45900000000000002,"15min":0.44500000000000001},"min":{"1min":0.314,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78000000000000003,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.57399999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.38200000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71899999999999997,"5min":1.042,"15min":1.042},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Jul 14 19:26:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.83299999999999996}]},{"osd":8,"last update":"Fri Jul 14 19:27:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.66000000000000003,"15min":0.64300000000000002},"min":{"1min":0.5,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.66000000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.93400000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.98699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":432670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.52000000000000002,"15min":0.48199999999999998},"min":{"1min":0.38,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.83399999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.505,"15min":0.47499999999999998},"min":{"1min":0.39900000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70199999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69499999999999995,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.307,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60899999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.373},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.42899999999999999},"min":{"1min":0.25900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.29199999999999998}]},{"osd":4,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80600000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.68899999999999995,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.34999999999999998}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.622,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71299999999999997,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66400000000000003,"15min":0.66500000000000004},"min":{"1min":0.54000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76900000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.438},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.46400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60199999999999998}]},{"osd":11,"last update":"Fri Jul 14 19:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58199999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":489172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.502,"15min":0.47399999999999998},"min":{"1min":0.42499999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.75,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.50900000000000001,"15min":0.47299999999999998},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.79400000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.441,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.85699999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.441},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.54700000000000004,"15min":0.51100000000000001},"min":{"1min":0.434,"5min":0.255,"15min":0.255},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.375}]},{"osd":3,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60799999999999998,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.9359999999999999,"5min":4.9359999999999999,"15min":4.9359999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.41699999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.46600000000000003,"15min":0.443},"min":{"1min":0.32300000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.51400000000000001},"min":{"1min":0.34100000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.7560000000000002,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.753}]},{"osd":9,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82199999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":431722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32100000000000001},"min":{"1min":0.28299999999999997,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55300000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.26400000000000001}]},{"osd":1,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68000000000000005,"5min":1.202,"15min":1.202},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.34000000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.58199999999999996},"min":{"1min":0.42799999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Jul 14 19:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.49399999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.77600000000000002,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.54300000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.438,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.624,"15min":0.622},"min":{"1min":0.45000000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54700000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78800000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.50800000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.622},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":901120,"data_stored":431788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.379,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.28899999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63800000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.434}]},{"osd":2,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.85399999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.48299999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.51500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53400000000000003}]},{"osd":8,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.879},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85899999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.745}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":1023170,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.48399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.78300000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.996,"15min":0.996},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66900000000000004,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68200000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.379}]},{"osd":3,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82999999999999996,"5min":1.022,"15min":1.022},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82999999999999996,"5min":1.044,"15min":1.044},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.623,"15min":0.62},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83299999999999996,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.623,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86499999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81299999999999994,"5min":1.087,"15min":1.087},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":1.113,"15min":1.113},"last":0.755}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.876,"5min":1.004,"15min":1.004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66600000000000004}]},{"osd":8,"last update":"Fri Jul 14 19:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91500000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60099999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.78900000000000003}]},{"osd":10,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.81100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":57980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:27:46.329 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-14T19:27:46.329 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-14T19:27:46.330 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-14T19:27:46.330 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph health --format=json 2023-07-14T19:27:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:46.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:46 smithi133 ceph-mon[128894]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:46.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:46 smithi082 ceph-mon[127461]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:46.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:46 smithi161 ceph-mon[129184]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:27:47.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:47 smithi082 ceph-mon[127461]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T19:27:47.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:47 smithi133 ceph-mon[128894]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T19:27:47.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:47 smithi161 ceph-mon[129184]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T19:27:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 -- 172.21.15.82:0/1373117316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 msgr2=0x7f97781037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 --2- 172.21.15.82:0/1373117316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781037b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f97740099c0 tx=0x7f9774009cd0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 -- 172.21.15.82:0/1373117316 shutdown_connections 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 --2- 172.21.15.82:0/1373117316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778103ee0 0x7f977810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 --2- 172.21.15.82:0/1373117316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97780fee40 0x7f97780ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 --2- 172.21.15.82:0/1373117316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781037b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 -- 172.21.15.82:0/1373117316 >> 172.21.15.82:0/1373117316 conn(0x7f9778075ca0 msgr2=0x7f97780760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 -- 172.21.15.82:0/1373117316 shutdown_connections 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.108+0000 7f977f2d3700 1 -- 172.21.15.82:0/1373117316 wait complete. 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 Processor -- start 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 -- start start 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97780fee40 0x7f9778194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778103ee0 0x7f97781985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778198ba0 con 0x7f97780ff7e0 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778198d10 con 0x7f97780fee40 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977f2d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9778198ed0 con 0x7f9778103ee0 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977c86e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977c86e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41362/0 (socket says 172.21.15.82:41362) 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977c86e700 1 -- 172.21.15.82:0/311149051 learned_addr learned my addr 172.21.15.82:0/311149051 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977d06f700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97780fee40 0x7f9778194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.109+0000 7f977d870700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778103ee0 0x7f97781985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977c86e700 1 -- 172.21.15.82:0/311149051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778103ee0 msgr2=0x7f97781985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977c86e700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778103ee0 0x7f97781985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977c86e700 1 -- 172.21.15.82:0/311149051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97780fee40 msgr2=0x7f9778194ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977c86e700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97780fee40 0x7f9778194ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977c86e700 1 -- 172.21.15.82:0/311149051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740095f0 con 0x7f97780ff7e0 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977c86e700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781951e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9774009990 tx=0x7f9774004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977401f070 con 0x7f97780ff7e0 2023-07-14T19:27:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9778199150 con 0x7f97780ff7e0 2023-07-14T19:27:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9774019cd0 con 0x7f97780ff7e0 2023-07-14T19:27:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740047a0 con 0x7f97780ff7e0 2023-07-14T19:27:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.110+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780682d0 con 0x7f97780ff7e0 2023-07-14T19:27:48.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.111+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977804e340 con 0x7f97780ff7e0 2023-07-14T19:27:48.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.112+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9774019810 con 0x7f97780ff7e0 2023-07-14T19:27:48.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.112+0000 7f976e7fc700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97640a11e0 0x7f97640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.113+0000 7f977d06f700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97640a11e0 0x7f97640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.113+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f97740cfde0 con 0x7f97780ff7e0 2023-07-14T19:27:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.113+0000 7f977d06f700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97640a11e0 0x7f97640a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9768005950 tx=0x7f976800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.116+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9774099580 con 0x7f97780ff7e0 2023-07-14T19:27:48.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.342+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f97780684f0 con 0x7f97780ff7e0 2023-07-14T19:27:48.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.342+0000 7f976e7fc700 1 -- 172.21.15.82:0/311149051 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f977409cba0 con 0x7f97780ff7e0 2023-07-14T19:27:48.344 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:48.344 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-14T19:27:48.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97640a11e0 msgr2=0x7f97640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97640a11e0 0x7f97640a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9768005950 tx=0x7f976800b410 comp rx=0 tx=0).stop 2023-07-14T19:27:48.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 msgr2=0x7f97781951e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781951e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9774009990 tx=0x7f9774004730 comp rx=0 tx=0).stop 2023-07-14T19:27:48.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 shutdown_connections 2023-07-14T19:27:48.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9778103ee0 0x7f97781985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97780fee40 0x7f9778194ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97640a11e0 0x7f97640a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 --2- 172.21.15.82:0/311149051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97780ff7e0 0x7f97781951e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 >> 172.21.15.82:0/311149051 conn(0x7f9778075ca0 msgr2=0x7f97780fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:48.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 shutdown_connections 2023-07-14T19:27:48.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.344+0000 7f977f2d3700 1 -- 172.21.15.82:0/311149051 wait complete. 2023-07-14T19:27:48.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:48 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/311149051' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T19:27:48.758 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-14T19:27:48.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:27:48.884 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- 172.21.15.82:0/1467374337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640ffcf0 msgr2=0x7f3d64100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/1467374337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640ffcf0 0x7f3d64100140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d50009990 tx=0x7f3d50009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- 172.21.15.82:0/1467374337 shutdown_connections 2023-07-14T19:27:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/1467374337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64100680 0x7f3d64106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/1467374337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d640feb00 0x7f3d640fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/1467374337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640ffcf0 0x7f3d64100140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.885 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- 172.21.15.82:0/1467374337 >> 172.21.15.82:0/1467374337 conn(0x7f3d640fa100 msgr2=0x7f3d640fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- 172.21.15.82:0/1467374337 shutdown_connections 2023-07-14T19:27:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- 172.21.15.82:0/1467374337 wait complete. 2023-07-14T19:27:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 Processor -- start 2023-07-14T19:27:48.886 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- start start 2023-07-14T19:27:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 0x7f3d64068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d640ffcf0 0x7f3d6419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64100680 0x7f3d6419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6419d3d0 con 0x7f3d640feb00 2023-07-14T19:27:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6419d540 con 0x7f3d640ffcf0 2023-07-14T19:27:48.888 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.884+0000 7f3d6af7d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6419d700 con 0x7f3d64100680 2023-07-14T19:27:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 0x7f3d64068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 0x7f3d64068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41372/0 (socket says 172.21.15.82:41372) 2023-07-14T19:27:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 -- 172.21.15.82:0/286807283 learned_addr learned my addr 172.21.15.82:0/286807283 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d63fff700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d640ffcf0 0x7f3d6419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d6951a700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64100680 0x7f3d6419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 -- 172.21.15.82:0/286807283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64100680 msgr2=0x7f3d6419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64100680 0x7f3d6419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 -- 172.21.15.82:0/286807283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d640ffcf0 msgr2=0x7f3d6419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:48.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d640ffcf0 0x7f3d6419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:48.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 -- 172.21.15.82:0/286807283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500095f0 con 0x7f3d640feb00 2023-07-14T19:27:48.890 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d68d19700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 0x7f3d64068370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c990 tx=0x7f3d5800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58007b00 con 0x7f3d640feb00 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d58007c60 con 0x7f3d640feb00 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6419d9e0 con 0x7f3d640feb00 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58018660 con 0x7f3d640feb00 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.885+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6419df40 con 0x7f3d640feb00 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.886+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d64105910 con 0x7f3d640feb00 2023-07-14T19:27:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.887+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3d5800f450 con 0x7f3d640feb00 2023-07-14T19:27:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.888+0000 7f3d61ffb700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3d540a12a0 0x7f3d540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.888+0000 7f3d63fff700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3d540a12a0 0x7f3d540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.888+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f3d580cc8c0 con 0x7f3d640feb00 2023-07-14T19:27:48.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.888+0000 7f3d63fff700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3d540a12a0 0x7f3d540a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b5c0 tx=0x7f3d50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:48.896 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:48.891+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3d580960e0 con 0x7f3d640feb00 2023-07-14T19:27:48.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:48 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/311149051' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T19:27:48.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:48 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/311149051' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-14T19:27:49.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.083+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d6402cb50 con 0x7f3d640feb00 2023-07-14T19:27:49.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.084+0000 7f3d61ffb700 1 -- 172.21.15.82:0/286807283 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f3d58099700 con 0x7f3d640feb00 2023-07-14T19:27:49.085 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:49.085 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:27:49.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3d540a12a0 msgr2=0x7f3d540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3d540a12a0 0x7f3d540a3750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b5c0 tx=0x7f3d50005fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 msgr2=0x7f3d64068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 0x7f3d64068370 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800c990 tx=0x7f3d5800cca0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 shutdown_connections 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d64100680 0x7f3d6419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3d540a12a0 0x7f3d540a3750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d640feb00 0x7f3d64068370 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 --2- 172.21.15.82:0/286807283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d640ffcf0 0x7f3d6419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 >> 172.21.15.82:0/286807283 conn(0x7f3d640fa100 msgr2=0x7f3d640fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 shutdown_connections 2023-07-14T19:27:49.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.086+0000 7f3d6af7d700 1 -- 172.21.15.82:0/286807283 wait complete. 2023-07-14T19:27:49.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:27:49.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:27:49.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 -- 172.21.15.82:0/4048426305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 msgr2=0x7f9920106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.225 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 --2- 172.21.15.82:0/4048426305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f9920106f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f991800b0a0 tx=0x7f991800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 -- 172.21.15.82:0/4048426305 shutdown_connections 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 --2- 172.21.15.82:0/4048426305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f9920106f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 --2- 172.21.15.82:0/4048426305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99200feb10 0x7f99200fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 --2- 172.21.15.82:0/4048426305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99200ffd00 0x7f9920100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.224+0000 7f99283bf700 1 -- 172.21.15.82:0/4048426305 >> 172.21.15.82:0/4048426305 conn(0x7f99200fa130 msgr2=0x7f99200fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 -- 172.21.15.82:0/4048426305 shutdown_connections 2023-07-14T19:27:49.226 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 -- 172.21.15.82:0/4048426305 wait complete. 2023-07-14T19:27:49.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 Processor -- start 2023-07-14T19:27:49.227 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 -- start start 2023-07-14T19:27:49.228 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99200feb10 0x7f9920195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99200ffd00 0x7f9920195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f992019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019d5b0 con 0x7f9920100690 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019d6f0 con 0x7f99200ffd00 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f99283bf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019d880 con 0x7f99200feb10 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f992695c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f992019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.229 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f992695c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f992019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41390/0 (socket says 172.21.15.82:41390) 2023-07-14T19:27:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f992695c700 1 -- 172.21.15.82:0/2803662080 learned_addr learned my addr 172.21.15.82:0/2803662080 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f992595a700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99200ffd00 0x7f9920195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.225+0000 7f992615b700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99200feb10 0x7f9920195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f992695c700 1 -- 172.21.15.82:0/2803662080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99200feb10 msgr2=0x7f9920195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.230 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f992695c700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99200feb10 0x7f9920195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.231 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f992695c700 1 -- 172.21.15.82:0/2803662080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99200ffd00 msgr2=0x7f9920195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f992695c700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99200ffd00 0x7f9920195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f992695c700 1 -- 172.21.15.82:0/2803662080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9918009d00 con 0x7f9920100690 2023-07-14T19:27:49.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f992695c700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f992019cfe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f991800b780 tx=0x7f991800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:49.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800e040 con 0x7f9920100690 2023-07-14T19:27:49.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9918004e60 con 0x7f9920100690 2023-07-14T19:27:49.232 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992019db00 con 0x7f9920100690 2023-07-14T19:27:49.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991801b6b0 con 0x7f9920100690 2023-07-14T19:27:49.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.226+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992019df80 con 0x7f9920100690 2023-07-14T19:27:49.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.227+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992010a2f0 con 0x7f9920100690 2023-07-14T19:27:49.233 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.228+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f991801b810 con 0x7f9920100690 2023-07-14T19:27:49.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.229+0000 7f99177fe700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f990c0a11e0 0x7f990c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.229+0000 7f992615b700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f990c0a11e0 0x7f990c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.229+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f99180ce250 con 0x7f9920100690 2023-07-14T19:27:49.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.229+0000 7f992615b700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f990c0a11e0 0x7f990c0a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99200ffb60 tx=0x7f991c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:49.234 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.233+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f99180978f0 con 0x7f9920100690 2023-07-14T19:27:49.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.424+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f992019e590 con 0x7f9920100690 2023-07-14T19:27:49.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.425+0000 7f99177fe700 1 -- 172.21.15.82:0/2803662080 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f991809af10 con 0x7f9920100690 2023-07-14T19:27:49.426 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:27:49.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f990c0a11e0 msgr2=0x7f990c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f990c0a11e0 0x7f990c0a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99200ffb60 tx=0x7f991c00a380 comp rx=0 tx=0).stop 2023-07-14T19:27:49.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 msgr2=0x7f992019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f992019cfe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f991800b780 tx=0x7f991800bd00 comp rx=0 tx=0).stop 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 shutdown_connections 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f990c0a11e0 0x7f990c0a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9920100690 0x7f992019cfe0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99200feb10 0x7f9920195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 --2- 172.21.15.82:0/2803662080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99200ffd00 0x7f9920195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 >> 172.21.15.82:0/2803662080 conn(0x7f99200fa130 msgr2=0x7f99200677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:49.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 shutdown_connections 2023-07-14T19:27:49.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.427+0000 7f99283bf700 1 -- 172.21.15.82:0/2803662080 wait complete. 2023-07-14T19:27:49.439 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:27:49.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 -- 172.21.15.82:0/3857107497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c240ffdc0 msgr2=0x7f7c24100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 --2- 172.21.15.82:0/3857107497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c240ffdc0 0x7f7c24100230 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7c140098f0 tx=0x7f7c14009c00 comp rx=0 tx=0).stop 2023-07-14T19:27:49.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 -- 172.21.15.82:0/3857107497 shutdown_connections 2023-07-14T19:27:49.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 --2- 172.21.15.82:0/3857107497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c24100770 0x7f7c24107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 --2- 172.21.15.82:0/3857107497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c240feb30 0x7f7c240fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 --2- 172.21.15.82:0/3857107497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c240ffdc0 0x7f7c24100230 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 -- 172.21.15.82:0/3857107497 >> 172.21.15.82:0/3857107497 conn(0x7f7c240fa150 msgr2=0x7f7c240fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:49.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 -- 172.21.15.82:0/3857107497 shutdown_connections 2023-07-14T19:27:49.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.569+0000 7f7c29456700 1 -- 172.21.15.82:0/3857107497 wait complete. 2023-07-14T19:27:49.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 Processor -- start 2023-07-14T19:27:49.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 -- start start 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c240feb30 0x7f7c24194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c240ffdc0 0x7f7c24194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 0x7f7c2419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2419c410 con 0x7f7c24100770 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2419c550 con 0x7f7c240ffdc0 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c29456700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2419c6e0 con 0x7f7c240feb30 2023-07-14T19:27:49.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 0x7f7c2419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 0x7f7c2419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41404/0 (socket says 172.21.15.82:41404) 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c237fe700 1 -- 172.21.15.82:0/1503033720 learned_addr learned my addr 172.21.15.82:0/1503033720 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c227fc700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c240ffdc0 0x7f7c24194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.570+0000 7f7c22ffd700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c240feb30 0x7f7c24194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c237fe700 1 -- 172.21.15.82:0/1503033720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c240feb30 msgr2=0x7f7c24194890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c237fe700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c240feb30 0x7f7c24194890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c237fe700 1 -- 172.21.15.82:0/1503033720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c240ffdc0 msgr2=0x7f7c24194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c237fe700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c240ffdc0 0x7f7c24194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c237fe700 1 -- 172.21.15.82:0/1503033720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180095f0 con 0x7f7c24100770 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c22ffd700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c240feb30 0x7f7c24194890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c237fe700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 0x7f7c2419be40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00ef30 tx=0x7f7c1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c018070 con 0x7f7c24100770 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c1c00f040 con 0x7f7c24100770 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c14009520 con 0x7f7c24100770 2023-07-14T19:27:49.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00cdb0 con 0x7f7c24100770 2023-07-14T19:27:49.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.571+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c24068bf0 con 0x7f7c24100770 2023-07-14T19:27:49.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.572+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c240695e0 con 0x7f7c24100770 2023-07-14T19:27:49.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.573+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7c1c003a00 con 0x7f7c24100770 2023-07-14T19:27:49.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.573+0000 7f7c0bfff700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c0c0a1260 0x7f7c0c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.574+0000 7f7c22ffd700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c0c0a1260 0x7f7c0c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.574+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f7c1c0ccfd0 con 0x7f7c24100770 2023-07-14T19:27:49.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.574+0000 7f7c22ffd700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c0c0a1260 0x7f7c0c0a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005fd0 tx=0x7f7c18009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:49.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.577+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7c1c096670 con 0x7f7c24100770 2023-07-14T19:27:49.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:49 smithi082 ceph-mon[127461]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:27:49.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:49 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/286807283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:49.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:49 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2803662080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:27:49.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.767+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c24105ba0 con 0x7f7c24100770 2023-07-14T19:27:49.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.767+0000 7f7c0bfff700 1 -- 172.21.15.82:0/1503033720 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c1c099c90 con 0x7f7c24100770 2023-07-14T19:27:49.768 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:27:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.769+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c0c0a1260 msgr2=0x7f7c0c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.769+0000 7f7c29456700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c0c0a1260 0x7f7c0c0a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005fd0 tx=0x7f7c18009310 comp rx=0 tx=0).stop 2023-07-14T19:27:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 msgr2=0x7f7c2419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 0x7f7c2419be40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00ef30 tx=0x7f7c1c00c3a0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 shutdown_connections 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f7c0c0a1260 0x7f7c0c0a3710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c24100770 0x7f7c2419be40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c240feb30 0x7f7c24194890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 --2- 172.21.15.82:0/1503033720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c240ffdc0 0x7f7c24194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 >> 172.21.15.82:0/1503033720 conn(0x7f7c240fa150 msgr2=0x7f7c24103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:49.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 shutdown_connections 2023-07-14T19:27:49.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.770+0000 7f7c29456700 1 -- 172.21.15.82:0/1503033720 wait complete. 2023-07-14T19:27:49.783 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:27:49.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 -- 172.21.15.82:0/1103588553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680feb10 msgr2=0x7f11680fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 --2- 172.21.15.82:0/1103588553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680feb10 0x7f11680fef20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1158009990 tx=0x7f1158009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 -- 172.21.15.82:0/1103588553 shutdown_connections 2023-07-14T19:27:49.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 --2- 172.21.15.82:0/1103588553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1168100690 0x7f1168106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 --2- 172.21.15.82:0/1103588553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680feb10 0x7f11680fef20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 --2- 172.21.15.82:0/1103588553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11680ffd00 0x7f1168100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 -- 172.21.15.82:0/1103588553 >> 172.21.15.82:0/1103588553 conn(0x7f11680fa130 msgr2=0x7f11680fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:49.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 -- 172.21.15.82:0/1103588553 shutdown_connections 2023-07-14T19:27:49.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 -- 172.21.15.82:0/1103588553 wait complete. 2023-07-14T19:27:49.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 Processor -- start 2023-07-14T19:27:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.907+0000 7f116d54a700 1 -- start start 2023-07-14T19:27:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11680feb10 0x7f1168102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 0x7f11681010e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1168100690 0x7f1168101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1168101dc0 con 0x7f11680ffd00 2023-07-14T19:27:49.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1168101f30 con 0x7f11680feb10 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116819fa30 con 0x7f1168100690 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 0x7f11681010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 0x7f11681010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41414/0 (socket says 172.21.15.82:41414) 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 -- 172.21.15.82:0/1870626297 learned_addr learned my addr 172.21.15.82:0/1870626297 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f1166ffd700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11680feb10 0x7f1168102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11677fe700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1168100690 0x7f1168101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 -- 172.21.15.82:0/1870626297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1168100690 msgr2=0x7f1168101620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1168100690 0x7f1168101620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 -- 172.21.15.82:0/1870626297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11680feb10 msgr2=0x7f1168102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11680feb10 0x7f1168102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 -- 172.21.15.82:0/1870626297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580095f0 con 0x7f11680ffd00 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f11667fc700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 0x7f11681010e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f115c00d950 tx=0x7f115c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c009940 con 0x7f11680ffd00 2023-07-14T19:27:49.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116819fd10 con 0x7f11680ffd00 2023-07-14T19:27:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115c010460 con 0x7f11680ffd00 2023-07-14T19:27:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c00f620 con 0x7f11680ffd00 2023-07-14T19:27:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.908+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11681a0210 con 0x7f11680ffd00 2023-07-14T19:27:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.909+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116810a2e0 con 0x7f11680ffd00 2023-07-14T19:27:49.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.910+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f115c0108d0 con 0x7f11680ffd00 2023-07-14T19:27:49.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.911+0000 7f114ffff700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f11500a11e0 0x7f11500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:49.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.911+0000 7f1166ffd700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f11500a11e0 0x7f11500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:49.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.911+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f115c0cc7a0 con 0x7f11680ffd00 2023-07-14T19:27:49.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.911+0000 7f1166ffd700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f11500a11e0 0x7f11500a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f115800b5c0 tx=0x7f1158005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:49.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:49.914+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f115c095eb0 con 0x7f11680ffd00 2023-07-14T19:27:49.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:49 smithi133 ceph-mon[128894]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:27:49.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:49 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/286807283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:49.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:49 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2803662080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:27:49.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:49 smithi161 ceph-mon[129184]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:27:49.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:49 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/286807283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:49.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:49 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2803662080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.103+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11681a08e0 con 0x7f11680ffd00 2023-07-14T19:27:50.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.104+0000 7f114ffff700 1 -- 172.21.15.82:0/1870626297 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f115c0160a0 con 0x7f11680ffd00 2023-07-14T19:27:50.105 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:27:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f11500a11e0 msgr2=0x7f11500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f11500a11e0 0x7f11500a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f115800b5c0 tx=0x7f1158005960 comp rx=0 tx=0).stop 2023-07-14T19:27:50.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 msgr2=0x7f11681010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 0x7f11681010e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f115c00d950 tx=0x7f115c00dc60 comp rx=0 tx=0).stop 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 shutdown_connections 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1168100690 0x7f1168101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11680feb10 0x7f1168102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f11500a11e0 0x7f11500a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 --2- 172.21.15.82:0/1870626297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11680ffd00 0x7f11681010e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 >> 172.21.15.82:0/1870626297 conn(0x7f11680fa130 msgr2=0x7f1168067800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:50.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 shutdown_connections 2023-07-14T19:27:50.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.106+0000 7f116d54a700 1 -- 172.21.15.82:0/1870626297 wait complete. 2023-07-14T19:27:50.119 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:27:50.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 -- 172.21.15.82:0/395088951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c100680 msgr2=0x7f222c106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 --2- 172.21.15.82:0/395088951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c100680 0x7f222c106f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f222400b0a0 tx=0x7f222400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 -- 172.21.15.82:0/395088951 shutdown_connections 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 --2- 172.21.15.82:0/395088951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c100680 0x7f222c106f40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 --2- 172.21.15.82:0/395088951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c0feb00 0x7f222c0fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 --2- 172.21.15.82:0/395088951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c0ffcf0 0x7f222c100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 -- 172.21.15.82:0/395088951 >> 172.21.15.82:0/395088951 conn(0x7f222c0fa100 msgr2=0x7f222c0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 -- 172.21.15.82:0/395088951 shutdown_connections 2023-07-14T19:27:50.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 -- 172.21.15.82:0/395088951 wait complete. 2023-07-14T19:27:50.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.239+0000 7f2231bdb700 1 Processor -- start 2023-07-14T19:27:50.242 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 -- start start 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c0feb00 0x7f222c1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 0x7f222c195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c100680 0x7f222c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19d540 con 0x7f222c0ffcf0 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19d680 con 0x7f222c0feb00 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f2231bdb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c19d810 con 0x7f222c100680 2023-07-14T19:27:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 0x7f222c195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 0x7f222c195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41428/0 (socket says 172.21.15.82:41428) 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 -- 172.21.15.82:0/3398222716 learned_addr learned my addr 172.21.15.82:0/3398222716 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222b7fe700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c0feb00 0x7f222c1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 -- 172.21.15.82:0/3398222716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c100680 msgr2=0x7f222c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c100680 0x7f222c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 -- 172.21.15.82:0/3398222716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c0feb00 msgr2=0x7f222c1959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c0feb00 0x7f222c1959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 -- 172.21.15.82:0/3398222716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009d00 con 0x7f222c0ffcf0 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.240+0000 7f222affd700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 0x7f222c195f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9a0 tx=0x7f221400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.241+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214009790 con 0x7f222c0ffcf0 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.241+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221400cbd0 con 0x7f222c0ffcf0 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.241+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c19daf0 con 0x7f222c0ffcf0 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.241+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214018680 con 0x7f222c0ffcf0 2023-07-14T19:27:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.241+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c19e000 con 0x7f222c0ffcf0 2023-07-14T19:27:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.242+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c10b4a0 con 0x7f222c0ffcf0 2023-07-14T19:27:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.243+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f221400cd40 con 0x7f222c0ffcf0 2023-07-14T19:27:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.243+0000 7f2228ff9700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f22180a1220 0x7f22180a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.243+0000 7f222b7fe700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f22180a1220 0x7f22180a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.243+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f2214014070 con 0x7f222c0ffcf0 2023-07-14T19:27:50.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.244+0000 7f222b7fe700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f22180a1220 0x7f22180a36d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f222c0ffb50 tx=0x7f221c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:50.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.247+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f22140965c0 con 0x7f222c0ffcf0 2023-07-14T19:27:50.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.478+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f222c105890 con 0x7f222c0ffcf0 2023-07-14T19:27:50.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.478+0000 7f2228ff9700 1 -- 172.21.15.82:0/3398222716 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f2214099be0 con 0x7f222c0ffcf0 2023-07-14T19:27:50.480 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:50.480 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:27:02.519960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:27:50.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f22180a1220 msgr2=0x7f22180a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f22180a1220 0x7f22180a36d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f222c0ffb50 tx=0x7f221c009260 comp rx=0 tx=0).stop 2023-07-14T19:27:50.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 msgr2=0x7f222c195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 0x7f222c195f00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9a0 tx=0x7f221400ecb0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 shutdown_connections 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f222c100680 0x7f222c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f222c0feb00 0x7f222c1959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f22180a1220 0x7f22180a36d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 --2- 172.21.15.82:0/3398222716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f222c0ffcf0 0x7f222c195f00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 >> 172.21.15.82:0/3398222716 conn(0x7f222c0fa100 msgr2=0x7f222c0fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:50.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 shutdown_connections 2023-07-14T19:27:50.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.481+0000 7f2231bdb700 1 -- 172.21.15.82:0/3398222716 wait complete. 2023-07-14T19:27:50.483 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:27:50.494 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:27:50.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 -- 172.21.15.82:0/4259450650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40feb20 msgr2=0x7f6bb40fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 --2- 172.21.15.82:0/4259450650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40feb20 0x7f6bb40fef30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009990 tx=0x7f6b9c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 -- 172.21.15.82:0/4259450650 shutdown_connections 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 --2- 172.21.15.82:0/4259450650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb41006a0 0x7f6bb4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 --2- 172.21.15.82:0/4259450650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40feb20 0x7f6bb40fef30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 --2- 172.21.15.82:0/4259450650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 -- 172.21.15.82:0/4259450650 >> 172.21.15.82:0/4259450650 conn(0x7f6bb40fa100 msgr2=0x7f6bb40fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 -- 172.21.15.82:0/4259450650 shutdown_connections 2023-07-14T19:27:50.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.615+0000 7f6bb889f700 1 -- 172.21.15.82:0/4259450650 wait complete. 2023-07-14T19:27:50.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 Processor -- start 2023-07-14T19:27:50.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 -- start start 2023-07-14T19:27:50.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bb40feb20 0x7f6bb4194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb41006a0 0x7f6bb419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb419c300 con 0x7f6bb40ffd10 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb419c440 con 0x7f6bb40feb20 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb889f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb419c5d0 con 0x7f6bb41006a0 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4194cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41442/0 (socket says 172.21.15.82:41442) 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.616+0000 7f6bb1d9b700 1 -- 172.21.15.82:0/3125523224 learned_addr learned my addr 172.21.15.82:0/3125523224 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:50.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb2d9d700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb41006a0 0x7f6bb419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb259c700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bb40feb20 0x7f6bb4194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb1d9b700 1 -- 172.21.15.82:0/3125523224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb41006a0 msgr2=0x7f6bb419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb1d9b700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb41006a0 0x7f6bb419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb1d9b700 1 -- 172.21.15.82:0/3125523224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bb40feb20 msgr2=0x7f6bb4194780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb1d9b700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bb40feb20 0x7f6bb4194780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb1d9b700 1 -- 172.21.15.82:0/3125523224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0095f0 con 0x7f6bb40ffd10 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb1d9b700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4194cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c920 tx=0x7f6ba400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:50.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004a30 con 0x7f6bb40ffd10 2023-07-14T19:27:50.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb419c8b0 con 0x7f6bb40ffd10 2023-07-14T19:27:50.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba40054e0 con 0x7f6bb40ffd10 2023-07-14T19:27:50.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4007c10 con 0x7f6bb40ffd10 2023-07-14T19:27:50.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.617+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40687d0 con 0x7f6bb40ffd10 2023-07-14T19:27:50.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.618+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb410a2f0 con 0x7f6bb40ffd10 2023-07-14T19:27:50.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.619+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6ba4005000 con 0x7f6bb40ffd10 2023-07-14T19:27:50.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.620+0000 7f6bab7fe700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6ba00a34b0 0x7f6ba00a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.620+0000 7f6bb259c700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6ba00a34b0 0x7f6ba00a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.620+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f6ba40cc980 con 0x7f6bb40ffd10 2023-07-14T19:27:50.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.620+0000 7f6bb259c700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6ba00a34b0 0x7f6ba00a5960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b5c0 tx=0x7f6b9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:50.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.623+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6ba4095f90 con 0x7f6bb40ffd10 2023-07-14T19:27:50.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.813+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bb410a510 con 0x7f6bb40ffd10 2023-07-14T19:27:50.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.815+0000 7f6bab7fe700 1 -- 172.21.15.82:0/3125523224 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f6ba401d090 con 0x7f6bb40ffd10 2023-07-14T19:27:50.816 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:50.817 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:27:50.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6ba00a34b0 msgr2=0x7f6ba00a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6ba00a34b0 0x7f6ba00a5960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00b5c0 tx=0x7f6b9c005960 comp rx=0 tx=0).stop 2023-07-14T19:27:50.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 msgr2=0x7f6bb4194cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4194cc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c920 tx=0x7f6ba400cc30 comp rx=0 tx=0).stop 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 shutdown_connections 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6bb41006a0 0x7f6bb419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bb40feb20 0x7f6bb4194780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6ba00a34b0 0x7f6ba00a5960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 --2- 172.21.15.82:0/3125523224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bb40ffd10 0x7f6bb4194cc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 >> 172.21.15.82:0/3125523224 conn(0x7f6bb40fa100 msgr2=0x7f6bb40fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:50.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 shutdown_connections 2023-07-14T19:27:50.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.817+0000 7f6bb889f700 1 -- 172.21.15.82:0/3125523224 wait complete. 2023-07-14T19:27:50.829 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:50 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1503033720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.829 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:50 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1870626297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.829 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:50 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3398222716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:50.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:27:50.831 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:27:50.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:50 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1503033720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:50 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1870626297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:50 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3398222716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:50.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 -- 172.21.15.82:0/1845168543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 msgr2=0x7f4650100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 --2- 172.21.15.82:0/1845168543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f4650100150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4638009990 tx=0x7f4638009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 -- 172.21.15.82:0/1845168543 shutdown_connections 2023-07-14T19:27:50.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 --2- 172.21.15.82:0/1845168543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4650100690 0x7f4650106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 --2- 172.21.15.82:0/1845168543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46500feb10 0x7f46500fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 --2- 172.21.15.82:0/1845168543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f4650100150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 -- 172.21.15.82:0/1845168543 >> 172.21.15.82:0/1845168543 conn(0x7f46500fa130 msgr2=0x7f46500fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 -- 172.21.15.82:0/1845168543 shutdown_connections 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.955+0000 7f4656519700 1 -- 172.21.15.82:0/1845168543 wait complete. 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 Processor -- start 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 -- start start 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46500feb10 0x7f4650068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f465019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4650100690 0x7f465019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465019d530 con 0x7f46500ffd00 2023-07-14T19:27:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465019d6a0 con 0x7f4650100690 2023-07-14T19:27:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4656519700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465019d860 con 0x7f46500feb10 2023-07-14T19:27:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f465019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f465019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41456/0 (socket says 172.21.15.82:41456) 2023-07-14T19:27:50.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 -- 172.21.15.82:0/2369909682 learned_addr learned my addr 172.21.15.82:0/2369909682 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f4654ab6700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4650100690 0x7f465019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464ffff700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46500feb10 0x7f4650068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 -- 172.21.15.82:0/2369909682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46500feb10 msgr2=0x7f4650068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46500feb10 0x7f4650068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 -- 172.21.15.82:0/2369909682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4650100690 msgr2=0x7f465019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4650100690 0x7f465019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:50.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 -- 172.21.15.82:0/2369909682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380095f0 con 0x7f46500ffd00 2023-07-14T19:27:50.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.956+0000 7f464f7fe700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f465019a710 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f46380047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:50.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.957+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463801d070 con 0x7f46500ffd00 2023-07-14T19:27:50.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.957+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4638022470 con 0x7f46500ffd00 2023-07-14T19:27:50.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.957+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465019dae0 con 0x7f46500ffd00 2023-07-14T19:27:50.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.957+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638017650 con 0x7f46500ffd00 2023-07-14T19:27:50.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.957+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465019df60 con 0x7f46500ffd00 2023-07-14T19:27:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.958+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4650106150 con 0x7f46500ffd00 2023-07-14T19:27:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.959+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f46380225e0 con 0x7f46500ffd00 2023-07-14T19:27:50.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.959+0000 7f464d7fa700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f463c0a1260 0x7f463c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.959+0000 7f464ffff700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f463c0a1260 0x7f463c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.959+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f46380cdf40 con 0x7f46500ffd00 2023-07-14T19:27:50.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.960+0000 7f464ffff700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f463c0a1260 0x7f463c0a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46500ffb60 tx=0x7f464000bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:50.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:50.963+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f46380976e0 con 0x7f46500ffd00 2023-07-14T19:27:50.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:50 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1503033720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:50 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1870626297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:27:50.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:50 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3398222716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.152+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f465002cb00 con 0x7f46500ffd00 2023-07-14T19:27:51.153 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.152+0000 7f464d7fa700 1 -- 172.21.15.82:0/2369909682 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f463809ad00 con 0x7f46500ffd00 2023-07-14T19:27:51.153 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:27:51.154 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f463c0a1260 msgr2=0x7f463c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f463c0a1260 0x7f463c0a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f46500ffb60 tx=0x7f464000bef0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 msgr2=0x7f465019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f465019a710 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f46380047a0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 shutdown_connections 2023-07-14T19:27:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4650100690 0x7f465019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46500feb10 0x7f4650068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f463c0a1260 0x7f463c0a3710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 --2- 172.21.15.82:0/2369909682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46500ffd00 0x7f465019a710 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 >> 172.21.15.82:0/2369909682 conn(0x7f46500fa130 msgr2=0x7f4650103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:51.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 shutdown_connections 2023-07-14T19:27:51.156 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.154+0000 7f4656519700 1 -- 172.21.15.82:0/2369909682 wait complete. 2023-07-14T19:27:51.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:27:51.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 -- 172.21.15.82:0/2223967881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97580ffd30 msgr2=0x7f9758100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.292 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 --2- 172.21.15.82:0/2223967881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97580ffd30 0x7f9758100180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9754009990 tx=0x7f9754009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 -- 172.21.15.82:0/2223967881 shutdown_connections 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 --2- 172.21.15.82:0/2223967881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97581006c0 0x7f9758106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 --2- 172.21.15.82:0/2223967881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97580feb40 0x7f97580fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 --2- 172.21.15.82:0/2223967881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97580ffd30 0x7f9758100180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 -- 172.21.15.82:0/2223967881 >> 172.21.15.82:0/2223967881 conn(0x7f97580fa160 msgr2=0x7f97580fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 -- 172.21.15.82:0/2223967881 shutdown_connections 2023-07-14T19:27:51.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 -- 172.21.15.82:0/2223967881 wait complete. 2023-07-14T19:27:51.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.291+0000 7f975f9f6700 1 Processor -- start 2023-07-14T19:27:51.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 -- start start 2023-07-14T19:27:51.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97580feb40 0x7f9758069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97580ffd30 0x7f975819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 0x7f975819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975819d500 con 0x7f97581006c0 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975819d670 con 0x7f97580feb40 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975819d830 con 0x7f97580ffd30 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 0x7f975819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 0x7f975819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41470/0 (socket says 172.21.15.82:41470) 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 -- 172.21.15.82:0/3690143763 learned_addr learned my addr 172.21.15.82:0/3690143763 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975cf91700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97580ffd30 0x7f975819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975d792700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97580feb40 0x7f9758069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 -- 172.21.15.82:0/3690143763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97580ffd30 msgr2=0x7f975819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97580ffd30 0x7f975819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 -- 172.21.15.82:0/3690143763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97580feb40 msgr2=0x7f9758069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97580feb40 0x7f9758069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 -- 172.21.15.82:0/3690143763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97540095f0 con 0x7f97581006c0 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975df93700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 0x7f975819cd30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9750007ee0 tx=0x7f975000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750017070 con 0x7f97581006c0 2023-07-14T19:27:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f975000f040 con 0x7f97581006c0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975819db10 con 0x7f97581006c0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.292+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750005740 con 0x7f97581006c0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.293+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975819dfd0 con 0x7f97581006c0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.294+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9758105a90 con 0x7f97581006c0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.294+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f975000dc60 con 0x7f97581006c0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.295+0000 7f974e7fc700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97440a11e0 0x7f97440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.295+0000 7f975d792700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97440a11e0 0x7f97440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.295+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f97500cc290 con 0x7f97581006c0 2023-07-14T19:27:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.296+0000 7f975d792700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97440a11e0 0x7f97440a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9748005950 tx=0x7f97480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:51.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.299+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f97500958a0 con 0x7f97581006c0 2023-07-14T19:27:51.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.489+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f975802cb00 con 0x7f97581006c0 2023-07-14T19:27:51.490 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.489+0000 7f974e7fc700 1 -- 172.21.15.82:0/3690143763 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9750095030 con 0x7f97581006c0 2023-07-14T19:27:51.490 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:27:51.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97440a11e0 msgr2=0x7f97440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97440a11e0 0x7f97440a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9748005950 tx=0x7f97480058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 msgr2=0x7f975819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 0x7f975819cd30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9750007ee0 tx=0x7f975000d3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 shutdown_connections 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f97440a11e0 0x7f97440a3690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97581006c0 0x7f975819cd30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97580feb40 0x7f9758069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 --2- 172.21.15.82:0/3690143763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97580ffd30 0x7f975819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.491+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 >> 172.21.15.82:0/3690143763 conn(0x7f97580fa160 msgr2=0x7f97580fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.492+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 shutdown_connections 2023-07-14T19:27:51.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.492+0000 7f975f9f6700 1 -- 172.21.15.82:0/3690143763 wait complete. 2023-07-14T19:27:51.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:27:51.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 -- 172.21.15.82:0/2744498739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c0fed40 msgr2=0x7f484c0ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 --2- 172.21.15.82:0/2744498739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c0fed40 0x7f484c0ff1b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4848009990 tx=0x7f4848009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 -- 172.21.15.82:0/2744498739 shutdown_connections 2023-07-14T19:27:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 --2- 172.21.15.82:0/2744498739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f484c0ff6f0 0x7f484c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 --2- 172.21.15.82:0/2744498739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484c100180 0x7f484c100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 --2- 172.21.15.82:0/2744498739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c0fed40 0x7f484c0ff1b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 -- 172.21.15.82:0/2744498739 >> 172.21.15.82:0/2744498739 conn(0x7f484c0f98a0 msgr2=0x7f484c0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 -- 172.21.15.82:0/2744498739 shutdown_connections 2023-07-14T19:27:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.630+0000 7f48541e1700 1 -- 172.21.15.82:0/2744498739 wait complete. 2023-07-14T19:27:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 Processor -- start 2023-07-14T19:27:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 -- start start 2023-07-14T19:27:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f484c0fed40 0x7f484c195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484c0ff6f0 0x7f484c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 0x7f484c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c19d4a0 con 0x7f484c100180 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c19d5e0 con 0x7f484c0ff6f0 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f48541e1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c19d770 con 0x7f484c0fed40 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 0x7f484c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 0x7f484c19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41484/0 (socket says 172.21.15.82:41484) 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 -- 172.21.15.82:0/2561255589 learned_addr learned my addr 172.21.15.82:0/2561255589 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:51.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485177c700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484c0ff6f0 0x7f484c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 -- 172.21.15.82:0/2561255589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f484c0fed40 msgr2=0x7f484c195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f484c0fed40 0x7f484c195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 -- 172.21.15.82:0/2561255589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484c0ff6f0 msgr2=0x7f484c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484c0ff6f0 0x7f484c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 -- 172.21.15.82:0/2561255589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480095f0 con 0x7f484c100180 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.631+0000 7f485277e700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 0x7f484c19ced0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f484400ada0 tx=0x7f484400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.632+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484400f070 con 0x7f484c100180 2023-07-14T19:27:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.632+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4844007b30 con 0x7f484c100180 2023-07-14T19:27:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.632+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c19da50 con 0x7f484c100180 2023-07-14T19:27:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.632+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48440053e0 con 0x7f484c100180 2023-07-14T19:27:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.632+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c19df60 con 0x7f484c100180 2023-07-14T19:27:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.632+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c10a320 con 0x7f484c100180 2023-07-14T19:27:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.634+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f484400c980 con 0x7f484c100180 2023-07-14T19:27:51.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.634+0000 7f4842ffd700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48380a11e0 0x7f48380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.634+0000 7f4851f7d700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48380a11e0 0x7f48380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.634+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f48440cc4f0 con 0x7f484c100180 2023-07-14T19:27:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.634+0000 7f4851f7d700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48380a11e0 0x7f48380a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f483c005fd0 tx=0x7f483c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.637+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4844095c90 con 0x7f484c100180 2023-07-14T19:27:51.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.830+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f484c19e570 con 0x7f484c100180 2023-07-14T19:27:51.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.831+0000 7f4842ffd700 1 -- 172.21.15.82:0/2561255589 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f484401a310 con 0x7f484c100180 2023-07-14T19:27:51.832 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:27:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48380a11e0 msgr2=0x7f48380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48380a11e0 0x7f48380a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f483c005fd0 tx=0x7f483c005d00 comp rx=0 tx=0).stop 2023-07-14T19:27:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 msgr2=0x7f484c19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 0x7f484c19ced0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f484400ada0 tx=0x7f484400c330 comp rx=0 tx=0).stop 2023-07-14T19:27:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 shutdown_connections 2023-07-14T19:27:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f48380a11e0 0x7f48380a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f484c100180 0x7f484c19ced0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f484c0fed40 0x7f484c195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 --2- 172.21.15.82:0/2561255589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484c0ff6f0 0x7f484c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 >> 172.21.15.82:0/2561255589 conn(0x7f484c0f98a0 msgr2=0x7f484c0fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:51.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 shutdown_connections 2023-07-14T19:27:51.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.833+0000 7f48541e1700 1 -- 172.21.15.82:0/2561255589 wait complete. 2023-07-14T19:27:51.844 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:51 smithi082 ceph-mon[127461]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:51.845 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:51 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3125523224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:51.845 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:51 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2369909682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:27:51.845 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:51 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3690143763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:27:51.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:27:51.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:51 smithi133 ceph-mon[128894]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:51.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:51 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3125523224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:51.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:51 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2369909682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:27:51.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:51 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3690143763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:27:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 -- 172.21.15.82:0/2519881152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 msgr2=0x7fda50100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 --2- 172.21.15.82:0/2519881152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda50100590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fda40009990 tx=0x7fda40009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 -- 172.21.15.82:0/2519881152 shutdown_connections 2023-07-14T19:27:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 --2- 172.21.15.82:0/2519881152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda500ff6f0 0x7fda50106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 --2- 172.21.15.82:0/2519881152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda50100590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 --2- 172.21.15.82:0/2519881152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda500fed40 0x7fda500ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 -- 172.21.15.82:0/2519881152 >> 172.21.15.82:0/2519881152 conn(0x7fda500f98a0 msgr2=0x7fda500fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.966+0000 7fda564e0700 1 -- 172.21.15.82:0/2519881152 shutdown_connections 2023-07-14T19:27:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 -- 172.21.15.82:0/2519881152 wait complete. 2023-07-14T19:27:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 Processor -- start 2023-07-14T19:27:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 -- start start 2023-07-14T19:27:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda500fed40 0x7fda50194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda500ff6f0 0x7fda50194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda5019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5019c3b0 con 0x7fda50100180 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5019c4f0 con 0x7fda500fed40 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.967+0000 7fda564e0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5019c680 con 0x7fda500ff6f0 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda5019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda5019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41496/0 (socket says 172.21.15.82:41496) 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 -- 172.21.15.82:0/2612455885 learned_addr learned my addr 172.21.15.82:0/2612455885 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda4ffff700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda500fed40 0x7fda50194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda4f7fe700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda500ff6f0 0x7fda50194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 -- 172.21.15.82:0/2612455885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda500ff6f0 msgr2=0x7fda50194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda500ff6f0 0x7fda50194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 -- 172.21.15.82:0/2612455885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda500fed40 msgr2=0x7fda50194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda500fed40 0x7fda50194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 -- 172.21.15.82:0/2612455885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400095f0 con 0x7fda50100180 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda54a7d700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda5019bde0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda48007ee0 tx=0x7fda4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48017070 con 0x7fda50100180 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5019c960 con 0x7fda50100180 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda4800f040 con 0x7fda50100180 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48005550 con 0x7fda50100180 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.968+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5019cec0 con 0x7fda50100180 2023-07-14T19:27:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.969+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5010a350 con 0x7fda50100180 2023-07-14T19:27:51.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.970+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fda4800db20 con 0x7fda50100180 2023-07-14T19:27:51.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.971+0000 7fda4d7fa700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fda380a1220 0x7fda380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:51.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.971+0000 7fda4ffff700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fda380a1220 0x7fda380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.971+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fda480cc060 con 0x7fda50100180 2023-07-14T19:27:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.971+0000 7fda4ffff700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fda380a1220 0x7fda380a36d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b5c0 tx=0x7fda40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:51.975+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fda48095770 con 0x7fda50100180 2023-07-14T19:27:51.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:51 smithi161 ceph-mon[129184]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:51.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:51 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3125523224' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:51.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:51 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2369909682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:27:51.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:51 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3690143763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:27:52.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.208+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda5010a540 con 0x7fda50100180 2023-07-14T19:27:52.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.208+0000 7fda4d7fa700 1 -- 172.21.15.82:0/2612455885 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10823 (secure 0 0 0) 0x7fda48098d90 con 0x7fda50100180 2023-07-14T19:27:52.209 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:52.209 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:27:02.519960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:27:52.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fda380a1220 msgr2=0x7fda380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fda380a1220 0x7fda380a36d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda4000b5c0 tx=0x7fda40005fd0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 msgr2=0x7fda5019bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda5019bde0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fda48007ee0 tx=0x7fda4800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 shutdown_connections 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fda380a1220 0x7fda380a36d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda50100180 0x7fda5019bde0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda500fed40 0x7fda50194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 --2- 172.21.15.82:0/2612455885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda500ff6f0 0x7fda50194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 >> 172.21.15.82:0/2612455885 conn(0x7fda500f98a0 msgr2=0x7fda500fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:52.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 shutdown_connections 2023-07-14T19:27:52.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.211+0000 7fda564e0700 1 -- 172.21.15.82:0/2612455885 wait complete. 2023-07-14T19:27:52.213 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:27:52.224 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:27:52.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 -- 172.21.15.82:0/3989270413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd398100f50 msgr2=0x7fd3981013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 --2- 172.21.15.82:0/3989270413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd398100f50 0x7fd3981013a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd384009990 tx=0x7fd384009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 -- 172.21.15.82:0/3989270413 shutdown_connections 2023-07-14T19:27:52.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 --2- 172.21.15.82:0/3989270413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3981018e0 0x7fd3981081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 --2- 172.21.15.82:0/3989270413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3980fdee0 0x7fd3980fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 --2- 172.21.15.82:0/3989270413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd398100f50 0x7fd3981013a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 -- 172.21.15.82:0/3989270413 >> 172.21.15.82:0/3989270413 conn(0x7fd3980f98a0 msgr2=0x7fd3980fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 -- 172.21.15.82:0/3989270413 shutdown_connections 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 -- 172.21.15.82:0/3989270413 wait complete. 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 Processor -- start 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.345+0000 7fd39edc3700 1 -- start start 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 0x7fd3981008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd398100f50 0x7fd3980fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3981018e0 0x7fd3980ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980ffbe0 con 0x7fd3980fdee0 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980ffd50 con 0x7fd398100f50 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39819f940 con 0x7fd3981018e0 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 0x7fd3981008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 0x7fd3981008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41508/0 (socket says 172.21.15.82:41508) 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 -- 172.21.15.82:0/1488497564 learned_addr learned my addr 172.21.15.82:0/1488497564 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:52.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd397fff700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd398100f50 0x7fd3980fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39d360700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3981018e0 0x7fd3980ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 -- 172.21.15.82:0/1488497564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3981018e0 msgr2=0x7fd3980ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3981018e0 0x7fd3980ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 -- 172.21.15.82:0/1488497564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd398100f50 msgr2=0x7fd3980fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd398100f50 0x7fd3980fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 -- 172.21.15.82:0/1488497564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840095f0 con 0x7fd3980fdee0 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39cb5f700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 0x7fd3981008c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00d980 tx=0x7fd38c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:52.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c004020 con 0x7fd3980fdee0 2023-07-14T19:27:52.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd38c00f690 con 0x7fd3980fdee0 2023-07-14T19:27:52.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38c003a60 con 0x7fd3980fdee0 2023-07-14T19:27:52.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.346+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39819fc20 con 0x7fd3980fdee0 2023-07-14T19:27:52.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.347+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3981a00e0 con 0x7fd3980fdee0 2023-07-14T19:27:52.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.347+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3981072f0 con 0x7fd3980fdee0 2023-07-14T19:27:52.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.348+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd38c00fb00 con 0x7fd3980fdee0 2023-07-14T19:27:52.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.349+0000 7fd395ffb700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3880a1220 0x7fd3880a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.349+0000 7fd397fff700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3880a1220 0x7fd3880a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.349+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fd38c0cc570 con 0x7fd3980fdee0 2023-07-14T19:27:52.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.349+0000 7fd397fff700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3880a1220 0x7fd3880a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b5c0 tx=0x7fd384005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:52.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.353+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd38c095c80 con 0x7fd3980fdee0 2023-07-14T19:27:52.546 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.546+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd39802caa0 con 0x7fd3980fdee0 2023-07-14T19:27:52.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.548+0000 7fd395ffb700 1 -- 172.21.15.82:0/1488497564 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7fd38c0992a0 con 0x7fd3980fdee0 2023-07-14T19:27:52.549 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:52.550 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:27:52.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3880a1220 msgr2=0x7fd3880a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3880a1220 0x7fd3880a36d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b5c0 tx=0x7fd384005fd0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 msgr2=0x7fd3981008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 0x7fd3981008c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00d980 tx=0x7fd38c00dc90 comp rx=0 tx=0).stop 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 shutdown_connections 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3981018e0 0x7fd3980ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3880a1220 0x7fd3880a36d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3980fdee0 0x7fd3981008c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 --2- 172.21.15.82:0/1488497564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd398100f50 0x7fd3980fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 >> 172.21.15.82:0/1488497564 conn(0x7fd3980f98a0 msgr2=0x7fd398104430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 shutdown_connections 2023-07-14T19:27:52.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.550+0000 7fd39edc3700 1 -- 172.21.15.82:0/1488497564 wait complete. 2023-07-14T19:27:52.563 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:27:52.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.683+0000 7f5cd0033700 1 -- 172.21.15.82:0/4140963417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 msgr2=0x7f5cc8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.683+0000 7f5cd0033700 1 --2- 172.21.15.82:0/4140963417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc8106f70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000b0a0 tx=0x7f5cc000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.683+0000 7f5cd0033700 1 -- 172.21.15.82:0/4140963417 shutdown_connections 2023-07-14T19:27:52.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.683+0000 7f5cd0033700 1 --2- 172.21.15.82:0/4140963417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc8106f70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.683+0000 7f5cd0033700 1 --2- 172.21.15.82:0/4140963417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc80feb30 0x7f5cc80fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.683+0000 7f5cd0033700 1 --2- 172.21.15.82:0/4140963417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc80ffd20 0x7f5cc8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- 172.21.15.82:0/4140963417 >> 172.21.15.82:0/4140963417 conn(0x7f5cc80fa150 msgr2=0x7f5cc80fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:52.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- 172.21.15.82:0/4140963417 shutdown_connections 2023-07-14T19:27:52.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- 172.21.15.82:0/4140963417 wait complete. 2023-07-14T19:27:52.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 Processor -- start 2023-07-14T19:27:52.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- start start 2023-07-14T19:27:52.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc80feb30 0x7f5cc8194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.688 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc80ffd20 0x7f5cc8194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819c3b0 con 0x7f5cc81006b0 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819c4f0 con 0x7f5cc80ffd20 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cd0033700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819c680 con 0x7f5cc80feb30 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cce5d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cce5d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41526/0 (socket says 172.21.15.82:41526) 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5cce5d0700 1 -- 172.21.15.82:0/3459367402 learned_addr learned my addr 172.21.15.82:0/3459367402 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:52.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.684+0000 7f5ccddcf700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc80feb30 0x7f5cc8194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5ccd5ce700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc80ffd20 0x7f5cc8194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cce5d0700 1 -- 172.21.15.82:0/3459367402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc80feb30 msgr2=0x7f5cc8194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cce5d0700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc80feb30 0x7f5cc8194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cce5d0700 1 -- 172.21.15.82:0/3459367402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc80ffd20 msgr2=0x7f5cc8194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cce5d0700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc80ffd20 0x7f5cc8194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cce5d0700 1 -- 172.21.15.82:0/3459367402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0009d00 con 0x7f5cc81006b0 2023-07-14T19:27:52.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cce5d0700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc819bde0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000b830 tx=0x7f5cc0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:52.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000e040 con 0x7f5cc81006b0 2023-07-14T19:27:52.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cc001ab90 con 0x7f5cc81006b0 2023-07-14T19:27:52.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc80689c0 con 0x7f5cc81006b0 2023-07-14T19:27:52.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00056f0 con 0x7f5cc81006b0 2023-07-14T19:27:52.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.685+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8068ed0 con 0x7f5cc81006b0 2023-07-14T19:27:52.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.686+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc80698c0 con 0x7f5cc81006b0 2023-07-14T19:27:52.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.687+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5cc001ad00 con 0x7f5cc81006b0 2023-07-14T19:27:52.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.687+0000 7f5cbeffd700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5cb40a1220 0x7f5cb40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:52.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.688+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f5cc00cfdf0 con 0x7f5cc81006b0 2023-07-14T19:27:52.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.688+0000 7f5ccddcf700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5cb40a1220 0x7f5cb40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:52.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.688+0000 7f5ccddcf700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5cb40a1220 0x7f5cb40a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80ffb80 tx=0x7f5cc400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:52.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.691+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5cc0099620 con 0x7f5cc81006b0 2023-07-14T19:27:52.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.925+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc8105930 con 0x7f5cc81006b0 2023-07-14T19:27:52.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.926+0000 7f5cbeffd700 1 -- 172.21.15.82:0/3459367402 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f5cc009cc40 con 0x7f5cc81006b0 2023-07-14T19:27:52.926 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:27:52.927 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:27:02.519960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:27:52.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5cb40a1220 msgr2=0x7f5cb40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5cb40a1220 0x7f5cb40a36d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80ffb80 tx=0x7f5cc400a3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 msgr2=0x7f5cc819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:52.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc819bde0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000b830 tx=0x7f5cc0004c80 comp rx=0 tx=0).stop 2023-07-14T19:27:52.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 shutdown_connections 2023-07-14T19:27:52.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5cb40a1220 0x7f5cb40a36d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cc81006b0 0x7f5cc819bde0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5cc80feb30 0x7f5cc8194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 --2- 172.21.15.82:0/3459367402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cc80ffd20 0x7f5cc8194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:52.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 >> 172.21.15.82:0/3459367402 conn(0x7f5cc80fa150 msgr2=0x7f5cc8103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:52.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 shutdown_connections 2023-07-14T19:27:52.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:52.928+0000 7f5cd0033700 1 -- 172.21.15.82:0/3459367402 wait complete. 2023-07-14T19:27:52.932 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:27:52.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:52 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2561255589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:27:52.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:52 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2612455885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:52.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:52 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1488497564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:52.940 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:52 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2561255589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:27:52.940 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:52 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2612455885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:52.940 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:52 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1488497564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:52.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T19:26:56.908603+0000', 'modified': '2023-07-14T19:27:02.519960+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24491, 'mds_2': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3902735752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3902735752}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3902735752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/3278917095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3278917095}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3278917095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/42384414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 42384414}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 42384414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.161:6839/4007435956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 4007435956}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 4007435956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6837/46070579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 46070579}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 46070579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6839/242970631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 242970631}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 242970631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T19:27:52.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T19:27:52.942 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T19:27:52.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:52 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2561255589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:27:52.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:52 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2612455885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:52.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:52 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1488497564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:27:53.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/702938161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef4102fb0 msgr2=0x7f2ef4105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/702938161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef4102fb0 0x7f2ef4105390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009990 tx=0x7f2ee8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/702938161 shutdown_connections 2023-07-14T19:27:53.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/702938161 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef41058d0 0x7f2ef4107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/702938161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ef4100690 0x7f2ef4102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/702938161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef4102fb0 0x7f2ef4105390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/702938161 >> 172.21.15.82:0/702938161 conn(0x7f2ef40fa130 msgr2=0x7f2ef40fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/702938161 shutdown_connections 2023-07-14T19:27:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.063+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/702938161 wait complete. 2023-07-14T19:27:53.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 Processor -- start 2023-07-14T19:27:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 -- start start 2023-07-14T19:27:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef4100690 0x7f2ef4195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ef4102fb0 0x7f2ef4195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 0x7f2ef419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef419d4b0 con 0x7f2ef41058d0 2023-07-14T19:27:53.067 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef419d5f0 con 0x7f2ef4102fb0 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef9bd3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef419d780 con 0x7f2ef4100690 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 0x7f2ef419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 0x7f2ef419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41540/0 (socket says 172.21.15.82:41540) 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef3fff700 1 -- 172.21.15.82:0/245625959 learned_addr learned my addr 172.21.15.82:0/245625959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ef4102fb0 0x7f2ef4195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.064+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef4100690 0x7f2ef4195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef3fff700 1 -- 172.21.15.82:0/245625959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef4100690 msgr2=0x7f2ef4195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef4100690 0x7f2ef4195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef3fff700 1 -- 172.21.15.82:0/245625959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ef4102fb0 msgr2=0x7f2ef4195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ef4102fb0 0x7f2ef4195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef3fff700 1 -- 172.21.15.82:0/245625959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80095f0 con 0x7f2ef41058d0 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef4100690 0x7f2ef4195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 0x7f2ef419cee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00ada0 tx=0x7f2eec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef0ff9700 1 -- 172.21.15.82:0/245625959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00f070 con 0x7f2ef41058d0 2023-07-14T19:27:53.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef0ff9700 1 -- 172.21.15.82:0/245625959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eec007b30 con 0x7f2ef41058d0 2023-07-14T19:27:53.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef0ff9700 1 -- 172.21.15.82:0/245625959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec005440 con 0x7f2ef41058d0 2023-07-14T19:27:53.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef419da60 con 0x7f2ef41058d0 2023-07-14T19:27:53.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.065+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef419df20 con 0x7f2ef41058d0 2023-07-14T19:27:53.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.067+0000 7f2ef0ff9700 1 -- 172.21.15.82:0/245625959 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2eec004ad0 con 0x7f2ef41058d0 2023-07-14T19:27:53.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.068+0000 7f2ef0ff9700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f2edc0a1260 0x7f2edc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.074 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.068+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f2edc0a1260 0x7f2edc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.068+0000 7f2ef0ff9700 1 -- 172.21.15.82:0/245625959 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f2eec0cbf90 con 0x7f2ef41058d0 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.068+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f2edc0a1260 0x7f2edc0a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005fd0 tx=0x7f2ee4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 Processor -- start 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 -- start start 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612f6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c612f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c611f5c60 con 0x561c612f5c90 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c612489f0 con 0x561c612f5320 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c61241e80 con 0x561c612f6850 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41548/0 (socket says 172.21.15.82:41548) 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/1823566725 learned_addr learned my addr 172.21.15.82:0/1823566725 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c612f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/1823566725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 msgr2=0x561c612f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c612f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/1823566725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 msgr2=0x561c612f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/1823566725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c6106fd90 con 0x561c612f5c90 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612f6270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009990 tx=0x7f2ee8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20895800afbc5651 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.077 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.069+0000 7f2ed9ffb700 1 -- 172.21.15.82:0/1823566725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800b9d0 con 0x561c612f5c90 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2ed9ffb700 1 -- 172.21.15.82:0/1823566725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee801e040 con 0x561c612f5c90 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2ed9ffb700 1 -- 172.21.15.82:0/1823566725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800f460 con 0x561c612f5c90 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1823566725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 msgr2=0x561c612f6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612f6270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009990 tx=0x7f2ee8009f60 comp rx=0 tx=0).stop 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1823566725 shutdown_connections 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c612f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.078 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1823566725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612f6270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1823566725 >> 172.21.15.82:0/1823566725 conn(0x561c612e6630 msgr2=0x561c612eb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1823566725 shutdown_connections 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.070+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1823566725 wait complete. 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 -- start start 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 Processor -- start 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c61306200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c611e1a70 con 0x561c612f5c90 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c611e1770 con 0x561c612f5320 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c610c58d0 con 0x561c612f6850 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2f05d6bb80 10 client.0 _ll_register_callbacks cb 0x561c611df710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41552/0 (socket says 172.21.15.82:41552) 2023-07-14T19:27:53.080 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/497600604 learned_addr learned my addr 172.21.15.82:0/497600604 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c61306200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 msgr2=0x561c61306200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c61306200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/497600604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 msgr2=0x561c612fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.071+0000 7f2ef2ffd700 1 -- 172.21.15.82:0/497600604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef4108920 con 0x561c612f5c90 2023-07-14T19:27:53.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612ff1c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800a010 tx=0x7f2ee800ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2166d17844e99a3f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ed2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/497600604 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8003d70 con 0x561c612f5c90 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/497600604 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f2ee800f460 con 0x561c612f5c90 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c6119b4a0 con 0x561c612f5c90 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/497600604 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ee800f620 con 0x561c612f5c90 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 10 client.15039 fetch_fsmap learned FSMap version 22 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561c61058c10 con 0x561c612f5c90 2023-07-14T19:27:53.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/497600604 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c6119b4a0 con 0x561c612f5c90 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/497600604 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2ee802b700 con 0x561c612f5c90 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 10 client.15039 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 10 client.15039 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 20 client.15039 populate_metadata read hostname 'smithi082' 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c613099f0 0x561c61309130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 4 client.15039 mds_command: new command op to 24481 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:27:53.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.072+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c612bfd50 con 0x561c613099f0 2023-07-14T19:27:53.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.073+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c613099f0 0x561c61309130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.073+0000 7f2ed2ffd700 10 client.15039 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:27:53.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.073+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c613099f0 0x561c61309130 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a43bfec52cb92999 server_cookie=d36ade064c84eabb in_seq=0 out_seq=0 2023-07-14T19:27:53.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/497600604 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561c612bfd50 con 0x561c613099f0 2023-07-14T19:27:53.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2ed2ffd700 10 client.15039 handle_command_reply: tid=0 2023-07-14T19:27:53.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 client.15039 shutdown 2023-07-14T19:27:53.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 msgr2=0x561c612ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612ff1c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800a010 tx=0x7f2ee800ba00 comp rx=0 tx=0).stop 2023-07-14T19:27:53.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 shutdown_connections 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c613099f0 0x561c61309130 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f6850 0x561c61306200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2ed2ffd700 0 client.15039 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/497600604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f5c90 0x561c612ff1c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 >> 172.21.15.82:0/497600604 conn(0x561c612e6630 msgr2=0x561c612ea600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 shutdown_connections 2023-07-14T19:27:53.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.074+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/497600604 wait complete. 2023-07-14T19:27:53.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.075+0000 7f2f05d6bb80 20 client.15039 trim_cache size 0 max 16384 2023-07-14T19:27:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.091+0000 7f2f05d6bb80 1 Processor -- start 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 -- start start 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c613061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c612eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c612bfd50 con 0x561c612f6850 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c6119b4a0 con 0x561c612f5320 2023-07-14T19:27:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c612fd6e0 con 0x561c612f5c90 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c612eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c612eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41562/0 (socket says 172.21.15.82:41562) 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1436592425 learned_addr learned my addr 172.21.15.82:0/1436592425 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1436592425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 msgr2=0x561c613061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c613061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1436592425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 msgr2=0x561c612e7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1436592425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c61096200 con 0x561c612f6850 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c612eb0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00a0a0 tx=0x7f2eec098fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15bdacf45893831 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2eda7fc700 1 -- 172.21.15.82:0/1436592425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec09d420 con 0x561c612f6850 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2eda7fc700 1 -- 172.21.15.82:0/1436592425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eec0a17b0 con 0x561c612f6850 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.092+0000 7f2eda7fc700 1 -- 172.21.15.82:0/1436592425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec09d240 con 0x561c612f6850 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1436592425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 msgr2=0x561c612eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c612eb0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00a0a0 tx=0x7f2eec098fb0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1436592425 shutdown_connections 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c612eb0a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c612e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1436592425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c613061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1436592425 >> 172.21.15.82:0/1436592425 conn(0x561c612e6630 msgr2=0x561c61309130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1436592425 shutdown_connections 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1436592425 wait complete. 2023-07-14T19:27:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 -- start start 2023-07-14T19:27:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.093+0000 7f2f05d6bb80 1 Processor -- start 2023-07-14T19:27:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c61301840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c61301d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c61304be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c61305120 con 0x561c612f6850 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c61305290 con 0x561c612f5320 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c61305450 con 0x561c612f5c90 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 10 client.0 _ll_register_callbacks cb 0x7f2ee8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c61304be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c61304be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41568/0 (socket says 172.21.15.82:41568) 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1382675533 learned_addr learned my addr 172.21.15.82:0/1382675533 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c61301840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef2ffd700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c61301d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 msgr2=0x561c61301d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c61301d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1382675533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 msgr2=0x561c61301840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c61301840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 -- 172.21.15.82:0/1382675533 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef419df20 con 0x561c612f6850 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c61301840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ef3fff700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c61304be0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2eec099380 tx=0x7f2eec09f220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f35d17a8ffb167 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ed2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/1382675533 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0988e0 con 0x561c612f6850 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/1382675533 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f2eec098a40 con 0x561c612f6850 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ee801e040 con 0x561c612f6850 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/1382675533 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eec0a1320 con 0x561c612f6850 2023-07-14T19:27:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 10 client.15045 fetch_fsmap learned FSMap version 22 2023-07-14T19:27:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.094+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561c6106e260 con 0x561c612f6850 2023-07-14T19:27:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/1382675533 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eec0c7080 con 0x561c612f6850 2023-07-14T19:27:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/1382675533 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2eec00c8a0 con 0x561c612f6850 2023-07-14T19:27:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2f05d6bb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:27:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2f05d6bb80 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T19:27:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2f05d6bb80 20 client.15045 populate_metadata read hostname 'smithi082' 2023-07-14T19:27:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c61362c60 0x561c6135f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2f05d6bb80 4 client.15045 mds_command: new command op to 24481 tid=0[{"prefix": "status"}] 2023-07-14T19:27:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561c6123ae60 con 0x561c61362c60 2023-07-14T19:27:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c61362c60 0x561c6135f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2ed2ffd700 10 client.15045 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:27:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.095+0000 7f2ef37fe700 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c61362c60 0x561c6135f680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43d1af03e72cadac server_cookie=ecbdf889a51bf938 in_seq=0 out_seq=0 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2ed2ffd700 1 -- 172.21.15.82:0/1382675533 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561c6123ae60 con 0x561c61362c60 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2ed2ffd700 10 client.15045 handle_command_reply: tid=0 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 client.15045 shutdown 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 msgr2=0x561c61304be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c61304be0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2eec099380 tx=0x7f2eec09f220 comp rx=0 tx=0).stop 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 shutdown_connections 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c612f6850 0x561c61304be0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561c612f5320 0x561c61301840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x561c61362c60 0x561c6135f680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 --2- 172.21.15.82:0/1382675533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c612f5c90 0x561c61301d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2ed2ffd700 0 client.15045 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 >> 172.21.15.82:0/1382675533 conn(0x561c612e6630 msgr2=0x561c612e7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 shutdown_connections 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 1 -- 172.21.15.82:0/1382675533 wait complete. 2023-07-14T19:27:53.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2f05d6bb80 20 client.15045 trim_cache size 0 max 16384 2023-07-14T19:27:53.108 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:27:53.108 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:27:53.108 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2023-07-14T19:27:53.108 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24481, 2023-07-14T19:27:53.108 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 51.324950354999999, 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 22, 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 68.620077621999997 2023-07-14T19:27:53.109 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.096+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f2edc0a1260 msgr2=0x7f2edc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f2edc0a1260 0x7f2edc0a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005fd0 tx=0x7f2ee4005ee0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 msgr2=0x7f2ef419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 0x7f2ef419cee0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00ada0 tx=0x7f2eec00c330 comp rx=0 tx=0).stop 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 shutdown_connections 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f2edc0a1260 0x7f2edc0a3710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ef41058d0 0x7f2ef419cee0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ef4100690 0x7f2ef4195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 --2- 172.21.15.82:0/245625959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ef4102fb0 0x7f2ef4195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 >> 172.21.15.82:0/245625959 conn(0x7f2ef40fa130 msgr2=0x7f2ef41012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 shutdown_connections 2023-07-14T19:27:53.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.097+0000 7f2ef9bd3700 1 -- 172.21.15.82:0/245625959 wait complete. 2023-07-14T19:27:53.112 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-14T19:27:53.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 -- 172.21.15.82:0/1856787416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca00ffd00 msgr2=0x7f1ca0100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/1856787416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca00ffd00 0x7f1ca0100150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009990 tx=0x7f1c9c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 -- 172.21.15.82:0/1856787416 shutdown_connections 2023-07-14T19:27:53.236 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/1856787416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca0100690 0x7f1ca0106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/1856787416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca00feb10 0x7f1ca00fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/1856787416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca00ffd00 0x7f1ca0100150 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 -- 172.21.15.82:0/1856787416 >> 172.21.15.82:0/1856787416 conn(0x7f1ca00fa130 msgr2=0x7f1ca00fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 -- 172.21.15.82:0/1856787416 shutdown_connections 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 -- 172.21.15.82:0/1856787416 wait complete. 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 Processor -- start 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.234+0000 7f1ca82b2700 1 -- start start 2023-07-14T19:27:53.237 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca00feb10 0x7f1ca0068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca00ffd00 0x7f1ca019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 0x7f1ca019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca019d480 con 0x7f1ca0100690 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca019d5f0 con 0x7f1ca00ffd00 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca019d7b0 con 0x7f1ca00feb10 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 0x7f1ca019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 0x7f1ca019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41582/0 (socket says 172.21.15.82:41582) 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 -- 172.21.15.82:0/3012285304 learned_addr learned my addr 172.21.15.82:0/3012285304 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca00feb10 0x7f1ca0068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca584d700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca00ffd00 0x7f1ca019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 -- 172.21.15.82:0/3012285304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca00feb10 msgr2=0x7f1ca0068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca00feb10 0x7f1ca0068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 -- 172.21.15.82:0/3012285304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca00ffd00 msgr2=0x7f1ca019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca00ffd00 0x7f1ca019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 -- 172.21.15.82:0/3012285304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0095f0 con 0x7f1ca0100690 2023-07-14T19:27:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca00feb10 0x7f1ca0068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca684f700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 0x7f1ca019cce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c98007d70 tx=0x7f1c9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1c977fe700 1 -- 172.21.15.82:0/3012285304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800ad40 con 0x7f1ca0100690 2023-07-14T19:27:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca019da90 con 0x7f1ca0100690 2023-07-14T19:27:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1c977fe700 1 -- 172.21.15.82:0/3012285304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c980128e0 con 0x7f1ca0100690 2023-07-14T19:27:53.240 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1c977fe700 1 -- 172.21.15.82:0/3012285304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98011780 con 0x7f1ca0100690 2023-07-14T19:27:53.241 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.235+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca019df50 con 0x7f1ca0100690 2023-07-14T19:27:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.237+0000 7f1c977fe700 1 -- 172.21.15.82:0/3012285304 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1c98012a50 con 0x7f1ca0100690 2023-07-14T19:27:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.238+0000 7f1c977fe700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c8c0a11e0 0x7f1c8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.238+0000 7f1c977fe700 1 -- 172.21.15.82:0/3012285304 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f1c980ccc30 con 0x7f1ca0100690 2023-07-14T19:27:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.238+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c8c0a11e0 0x7f1c8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.238+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c8c0a11e0 0x7f1c8c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1c90005950 tx=0x7f1c900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 Processor -- start 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 -- start start 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf4b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf4be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf4c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943be4fc60 con 0x55943bf4af10 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bea29f0 con 0x55943bf4b860 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943be9be80 con 0x55943bf4c460 2023-07-14T19:27:53.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.239+0000 7f1ca604e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf4b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf4b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41586/0 (socket says 172.21.15.82:41586) 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 -- 172.21.15.82:0/680061285 learned_addr learned my addr 172.21.15.82:0/680061285 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca584d700 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf4be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca684f700 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf4c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 -- 172.21.15.82:0/680061285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 msgr2=0x55943bf4c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf4c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 -- 172.21.15.82:0/680061285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 msgr2=0x55943bf4be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf4be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 -- 172.21.15.82:0/680061285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943bcc9d90 con 0x55943bf4af10 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1ca604e700 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf4b320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000ff00 tx=0x7f1c90012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13e360f91a851fa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1c94daa700 1 -- 172.21.15.82:0/680061285 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90011c60 con 0x55943bf4af10 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1c94daa700 1 -- 172.21.15.82:0/680061285 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c90011dc0 con 0x55943bf4af10 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.240+0000 7f1c94daa700 1 -- 172.21.15.82:0/680061285 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001d680 con 0x55943bf4af10 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- 172.21.15.82:0/680061285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 msgr2=0x55943bf4b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf4b320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000ff00 tx=0x7f1c90012000 comp rx=0 tx=0).stop 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- 172.21.15.82:0/680061285 shutdown_connections 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf4b320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/680061285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf4be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- 172.21.15.82:0/680061285 >> 172.21.15.82:0/680061285 conn(0x55943bf40630 msgr2=0x55943bf45980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- 172.21.15.82:0/680061285 shutdown_connections 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- 172.21.15.82:0/680061285 wait complete. 2023-07-14T19:27:53.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- start start 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 Processor -- start 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf54e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bf5be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943be3ba70 con 0x55943bf4c460 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943be3b770 con 0x55943bf4af10 2023-07-14T19:27:53.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.241+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bd1f8d0 con 0x55943bf4b860 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1cb444ab80 10 client.0 _ll_register_callbacks cb 0x55943be39710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bf5be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bf5be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41596/0 (socket says 172.21.15.82:41596) 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 -- 172.21.15.82:0/3932389130 learned_addr learned my addr 172.21.15.82:0/3932389130 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca584d700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf54e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 -- 172.21.15.82:0/3932389130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 msgr2=0x55943bf54e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf54e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 -- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 msgr2=0x55943bf548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 -- 172.21.15.82:0/3932389130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9000fbb0 con 0x55943bf4c460 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1ca684f700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bf5be70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1c98096b80 tx=0x7f1c98019450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4436320f7349b38c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1c89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1c89ffb700 1 -- 172.21.15.82:0/3932389130 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980a2560 con 0x55943bf4c460 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1c89ffb700 1 -- 172.21.15.82:0/3932389130 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f1c98099600 con 0x55943bf4c460 2023-07-14T19:27:53.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55943bdf54a0 con 0x55943bf4c460 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1c89ffb700 1 -- 172.21.15.82:0/3932389130 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c980b8070 con 0x55943bf4c460 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1cb444ab80 10 client.15057 fetch_fsmap learned FSMap version 22 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55943bcb2c10 con 0x55943bf4c460 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.242+0000 7f1c89ffb700 1 -- 172.21.15.82:0/3932389130 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55943bdf54a0 con 0x55943bf4c460 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1c89ffb700 1 -- 172.21.15.82:0/3932389130 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1c980b82f0 con 0x55943bf4c460 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1cb444ab80 10 client.15057 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:27:53.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1cb444ab80 10 client.15057 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1cb444ab80 20 client.15057 populate_metadata read hostname 'smithi082' 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bf5fb90 0x55943bf5efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1cb444ab80 4 client.15057 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55943bf19d50 con 0x55943bf5fb90 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bf5fb90 0x55943bf5efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1c89ffb700 10 client.15057 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.243+0000 7f1ca604e700 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bf5fb90 0x55943bf5efd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e91ec575a1f3f5b9 server_cookie=effaf2fcc8e57c6e in_seq=0 out_seq=0 2023-07-14T19:27:53.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1c89ffb700 1 -- 172.21.15.82:0/3932389130 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55943bf19d50 con 0x55943bf5fb90 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1c89ffb700 10 client.15057 handle_command_reply: tid=0 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 client.15057 shutdown 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 msgr2=0x55943bf5be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bf5be70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1c98096b80 tx=0x7f1c98019450 comp rx=0 tx=0).stop 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 shutdown_connections 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bf5fb90 0x55943bf5efd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bf5be70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1c89ffb700 0 client.15057 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:27:53.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/3932389130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf54e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 >> 172.21.15.82:0/3932389130 conn(0x55943bf40630 msgr2=0x55943bf44600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 shutdown_connections 2023-07-14T19:27:53.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.244+0000 7f1cb444ab80 1 -- 172.21.15.82:0/3932389130 wait complete. 2023-07-14T19:27:53.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.245+0000 7f1cb444ab80 20 client.15057 trim_cache size 0 max 16384 2023-07-14T19:27:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 Processor -- start 2023-07-14T19:27:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 -- start start 2023-07-14T19:27:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf5c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bf19d50 con 0x55943bf4af10 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bdf54a0 con 0x55943bf4b860 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943be6a790 con 0x55943bf4c460 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41600/0 (socket says 172.21.15.82:41600) 2023-07-14T19:27:53.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 -- 172.21.15.82:0/650860285 learned_addr learned my addr 172.21.15.82:0/650860285 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca584d700 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca684f700 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf5c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 -- 172.21.15.82:0/650860285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 msgr2=0x55943bf5c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf5c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 -- 172.21.15.82:0/650860285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 msgr2=0x55943bf5be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 -- 172.21.15.82:0/650860285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943bcf0200 con 0x55943bf4af10 2023-07-14T19:27:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca684f700 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf5c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.262+0000 7f1ca604e700 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf450a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1c90024000 tx=0x7f1c900046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c223d2e2735e00c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1c955ab700 1 -- 172.21.15.82:0/650860285 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90035250 con 0x55943bf4af10 2023-07-14T19:27:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1c955ab700 1 -- 172.21.15.82:0/650860285 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c9002e040 con 0x55943bf4af10 2023-07-14T19:27:53.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1c955ab700 1 -- 172.21.15.82:0/650860285 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001e7c0 con 0x55943bf4af10 2023-07-14T19:27:53.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 -- 172.21.15.82:0/650860285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 msgr2=0x55943bf450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf450a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1c90024000 tx=0x7f1c900046f0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 -- 172.21.15.82:0/650860285 shutdown_connections 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4c460 0x55943bf5c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4af10 0x55943bf450a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/650860285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 -- 172.21.15.82:0/650860285 >> 172.21.15.82:0/650860285 conn(0x55943bf40630 msgr2=0x55943bf41b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 -- 172.21.15.82:0/650860285 shutdown_connections 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 -- 172.21.15.82:0/650860285 wait complete. 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.263+0000 7f1cb444ab80 1 -- start start 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 Processor -- start 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf51a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bfb37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bfb3d30 con 0x55943bf4c460 2023-07-14T19:27:53.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bfb3e70 con 0x55943bf4af10 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943bfb3fb0 con 0x55943bf4b860 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1cb444ab80 10 client.0 _ll_register_callbacks cb 0x7f1c98012d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bfb37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bfb37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41610/0 (socket says 172.21.15.82:41610) 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 -- 172.21.15.82:0/2828044699 learned_addr learned my addr 172.21.15.82:0/2828044699 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca604e700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf51a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca584d700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 -- 172.21.15.82:0/2828044699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 msgr2=0x55943bf5be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 -- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 msgr2=0x55943bf51a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf51a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 -- 172.21.15.82:0/2828044699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55943bcc9d90 con 0x55943bf4c460 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca584d700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1ca684f700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bfb37f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800afd0 tx=0x7f1c98012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928d1f8693ef523f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.264+0000 7f1c89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1c89ffb700 1 -- 172.21.15.82:0/2828044699 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980b8040 con 0x55943bf4c460 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c90011dc0 con 0x55943bf4c460 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1c89ffb700 1 -- 172.21.15.82:0/2828044699 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f1c980b2d00 con 0x55943bf4c460 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1c89ffb700 1 -- 172.21.15.82:0/2828044699 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c98096450 con 0x55943bf4c460 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 10 client.15063 fetch_fsmap learned FSMap version 22 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55943bcc8260 con 0x55943bf4c460 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1c89ffb700 1 -- 172.21.15.82:0/2828044699 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c90011dc0 con 0x55943bf4c460 2023-07-14T19:27:53.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1c89ffb700 1 -- 172.21.15.82:0/2828044699 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1c980b82c0 con 0x55943bf4c460 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 10 client.15063 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 10 client.15063 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 20 client.15063 populate_metadata read hostname 'smithi082' 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bfb88b0 0x55943bfb5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 4 client.15063 mds_command: new command op to 24491 tid=0[{"prefix": "status"}] 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55943be94e60 con 0x55943bfb88b0 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.265+0000 7f1ca604e700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bfb88b0 0x55943bfb5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.275 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1c89ffb700 10 client.15063 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1ca604e700 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bfb88b0 0x55943bfb5260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=640dc82b796bcdf3 server_cookie=e31bbe32585c4b8b in_seq=0 out_seq=0 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1c89ffb700 1 -- 172.21.15.82:0/2828044699 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55943be94e60 con 0x55943bfb88b0 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1c89ffb700 10 client.15063 handle_command_reply: tid=0 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 client.15063 shutdown 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 msgr2=0x55943bfb37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bfb37f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800afd0 tx=0x7f1c98012f20 comp rx=0 tx=0).stop 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 shutdown_connections 2023-07-14T19:27:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55943bfb88b0 0x55943bfb5260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55943bf4c460 0x55943bfb37f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1c89ffb700 0 client.15063 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55943bf4af10 0x55943bf51a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 --2- 172.21.15.82:0/2828044699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55943bf4b860 0x55943bf5be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 >> 172.21.15.82:0/2828044699 conn(0x55943bf40630 msgr2=0x55943bf450a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 shutdown_connections 2023-07-14T19:27:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 1 -- 172.21.15.82:0/2828044699 wait complete. 2023-07-14T19:27:53.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.266+0000 7f1cb444ab80 20 client.15063 trim_cache size 0 max 16384 2023-07-14T19:27:53.278 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:27:53.278 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:27:53.278 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2023-07-14T19:27:53.278 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24491, 2023-07-14T19:27:53.278 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 52.498322735999999, 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 22, 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-07-14T19:27:53.279 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 70.945127056999993 2023-07-14T19:27:53.280 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c8c0a11e0 msgr2=0x7f1c8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c8c0a11e0 0x7f1c8c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1c90005950 tx=0x7f1c900058e0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 msgr2=0x7f1ca019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 0x7f1ca019cce0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c98007d70 tx=0x7f1c9800a340 comp rx=0 tx=0).stop 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 shutdown_connections 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1c8c0a11e0 0x7f1c8c0a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ca0100690 0x7f1ca019cce0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1ca00feb10 0x7f1ca0068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 --2- 172.21.15.82:0/3012285304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ca00ffd00 0x7f1ca019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 >> 172.21.15.82:0/3012285304 conn(0x7f1ca00fa130 msgr2=0x7f1ca00fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 shutdown_connections 2023-07-14T19:27:53.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.267+0000 7f1ca82b2700 1 -- 172.21.15.82:0/3012285304 wait complete. 2023-07-14T19:27:53.282 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T19:27:53.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.406+0000 7fbc42f71700 1 -- 172.21.15.82:0/526907093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1018e0 msgr2=0x7fbc3c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.406+0000 7fbc42f71700 1 --2- 172.21.15.82:0/526907093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1018e0 0x7fbc3c1081a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b0a0 tx=0x7fbc3400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- 172.21.15.82:0/526907093 shutdown_connections 2023-07-14T19:27:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 --2- 172.21.15.82:0/526907093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c1018e0 0x7fbc3c1081a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 --2- 172.21.15.82:0/526907093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c0fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 --2- 172.21.15.82:0/526907093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc3c100f50 0x7fbc3c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- 172.21.15.82:0/526907093 >> 172.21.15.82:0/526907093 conn(0x7fbc3c0f98a0 msgr2=0x7fbc3c0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- 172.21.15.82:0/526907093 shutdown_connections 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- 172.21.15.82:0/526907093 wait complete. 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 Processor -- start 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- start start 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c1008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c100f50 0x7fbc3c0fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc3c1018e0 0x7fbc3c0ff440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0ffbe0 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c0ffd50 con 0x7fbc3c1018e0 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc42f71700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c19fac0 con 0x7fbc3c100f50 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc40d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c1008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc40d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c1008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41618/0 (socket says 172.21.15.82:41618) 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.407+0000 7fbc40d0d700 1 -- 172.21.15.82:0/514282404 learned_addr learned my addr 172.21.15.82:0/514282404 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc4150e700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc3c1018e0 0x7fbc3c0ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c100f50 0x7fbc3c0fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc40d0d700 1 -- 172.21.15.82:0/514282404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c100f50 msgr2=0x7fbc3c0fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c100f50 0x7fbc3c0fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc40d0d700 1 -- 172.21.15.82:0/514282404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc3c1018e0 msgr2=0x7fbc3c0ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc3c1018e0 0x7fbc3c0ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc40d0d700 1 -- 172.21.15.82:0/514282404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34009d00 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c100f50 0x7fbc3c0fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:53.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c1008c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d980 tx=0x7fbc3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc39ffb700 1 -- 172.21.15.82:0/514282404 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30004020 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc39ffb700 1 -- 172.21.15.82:0/514282404 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc30009730 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c19fda0 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.408+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c1a01f0 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.409+0000 7fbc39ffb700 1 -- 172.21.15.82:0/514282404 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30004380 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.410+0000 7fbc39ffb700 1 -- 172.21.15.82:0/514282404 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbc30009b80 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.411+0000 7fbc39ffb700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbc2c0a11e0 0x7fbc2c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.411+0000 7fbc39ffb700 1 -- 172.21.15.82:0/514282404 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fbc300cc680 con 0x7fbc3c0fdee0 2023-07-14T19:27:53.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.411+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbc2c0a11e0 0x7fbc2c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.411+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbc2c0a11e0 0x7fbc2c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800bcf0 tx=0x7fbc2800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 Processor -- start 2023-07-14T19:27:53.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 -- start start 2023-07-14T19:27:53.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8ef5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8ef5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8ef68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8df9c60 con 0x5604e8ef6460 2023-07-14T19:27:53.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8e4c9f0 con 0x5604e8ef4f10 2023-07-14T19:27:53.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8e45e80 con 0x5604e8ef5860 2023-07-14T19:27:53.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8ef68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8ef68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41630/0 (socket says 172.21.15.82:41630) 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 -- 172.21.15.82:0/1625067298 learned_addr learned my addr 172.21.15.82:0/1625067298 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8ef5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8ef5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 -- 172.21.15.82:0/1625067298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 msgr2=0x5604e8ef5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8ef5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 -- 172.21.15.82:0/1625067298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 msgr2=0x5604e8ef5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8ef5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 -- 172.21.15.82:0/1625067298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e8c73d90 con 0x5604e8ef6460 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8ef5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc4150e700 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8ef68d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbc34015000 tx=0x7fbc340074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea619347ae9d9d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.412+0000 7fbc26ffd700 1 -- 172.21.15.82:0/1625067298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400e040 con 0x5604e8ef6460 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc26ffd700 1 -- 172.21.15.82:0/1625067298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc340060c0 con 0x5604e8ef6460 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc26ffd700 1 -- 172.21.15.82:0/1625067298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3401c940 con 0x5604e8ef6460 2023-07-14T19:27:53.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 -- 172.21.15.82:0/1625067298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 msgr2=0x5604e8ef68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8ef68d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbc34015000 tx=0x7fbc340074e0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 -- 172.21.15.82:0/1625067298 shutdown_connections 2023-07-14T19:27:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8ef68d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8ef5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/1625067298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8ef5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 -- 172.21.15.82:0/1625067298 >> 172.21.15.82:0/1625067298 conn(0x5604e8eea630 msgr2=0x5604e8eef980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 -- 172.21.15.82:0/1625067298 shutdown_connections 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 -- 172.21.15.82:0/1625067298 wait complete. 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.413+0000 7fbc4f109b80 1 -- start start 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 Processor -- start 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 0x5604e8efe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8efedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f05e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8de5a70 con 0x5604e8ef4f10 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8de5770 con 0x5604e8ef6460 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8cc98d0 con 0x5604e8ef5860 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4f109b80 10 client.0 _ll_register_callbacks cb 0x5604e8de3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 0x5604e8efe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 0x5604e8efe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41632/0 (socket says 172.21.15.82:41632) 2023-07-14T19:27:53.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 -- 172.21.15.82:0/3732002008 learned_addr learned my addr 172.21.15.82:0/3732002008 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8efedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4150e700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f05e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 -- 172.21.15.82:0/3732002008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 msgr2=0x5604e8efedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8efedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 -- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 msgr2=0x5604e8f05e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f05e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc40d0d700 1 -- 172.21.15.82:0/3732002008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c108960 con 0x5604e8ef4f10 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.414+0000 7fbc4150e700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f05e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 0x5604e8efe890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbc30099750 tx=0x7fbc3009da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3bbbd20e63d2ac server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3732002008 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30099140 con 0x5604e8ef4f10 2023-07-14T19:27:53.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3732002008 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fbc300b2200 con 0x5604e8ef4f10 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604e8d9f4a0 con 0x5604e8ef4f10 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3732002008 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc300a1a10 con 0x5604e8ef4f10 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 10 client.15075 fetch_fsmap learned FSMap version 22 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5604e8c5cc10 con 0x5604e8ef4f10 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3732002008 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604e8d9f4a0 con 0x5604e8ef4f10 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3732002008 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc300b9100 con 0x5604e8ef4f10 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:27:53.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 20 client.15075 populate_metadata read hostname 'smithi082' 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f09a00 0x5604e8f08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 4 client.15075 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.415+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604e8ec3d50 con 0x5604e8f09a00 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.416+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f09a00 0x5604e8f08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.416+0000 7fbc1bfff700 10 client.15075 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:27:53.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.416+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f09a00 0x5604e8f08ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff91ffa19628b7d server_cookie=4bfc9fbabcc3eff3 in_seq=0 out_seq=0 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3732002008 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5604e8ec3d50 con 0x5604e8f09a00 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc1bfff700 10 client.15075 handle_command_reply: tid=0 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 client.15075 shutdown 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 msgr2=0x5604e8efe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 0x5604e8efe890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbc30099750 tx=0x7fbc3009da80 comp rx=0 tx=0).stop 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 shutdown_connections 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f05e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef4f10 0x5604e8efe890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f09a00 0x5604e8f08ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3732002008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8efedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc1bfff700 0 client.15075 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 >> 172.21.15.82:0/3732002008 conn(0x5604e8eea630 msgr2=0x5604e8eee600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 shutdown_connections 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3732002008 wait complete. 2023-07-14T19:27:53.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.417+0000 7fbc4f109b80 20 client.15075 trim_cache size 0 max 16384 2023-07-14T19:27:53.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 Processor -- start 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 -- start start 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef4f10 0x5604e8f05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 0x5604e8f12790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8cd5de0 con 0x5604e8ef5860 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.434+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8e04480 con 0x5604e8ef6460 2023-07-14T19:27:53.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8de1d50 con 0x5604e8ef4f10 2023-07-14T19:27:53.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 0x5604e8f12790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 0x5604e8f12790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41642/0 (socket says 172.21.15.82:41642) 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 -- 172.21.15.82:0/2920764450 learned_addr learned my addr 172.21.15.82:0/2920764450 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef4f10 0x5604e8f05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 -- 172.21.15.82:0/2920764450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef4f10 msgr2=0x5604e8f05da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef4f10 0x5604e8f05da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 -- 172.21.15.82:0/2920764450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 msgr2=0x5604e8f08ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 -- 172.21.15.82:0/2920764450 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e8c9a200 con 0x5604e8ef5860 2023-07-14T19:27:53.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 0x5604e8f12790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800b600 tx=0x7fbc28015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce277417c80916 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc277fe700 1 -- 172.21.15.82:0/2920764450 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28011bb0 con 0x5604e8ef5860 2023-07-14T19:27:53.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc277fe700 1 -- 172.21.15.82:0/2920764450 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc28011d10 con 0x5604e8ef5860 2023-07-14T19:27:53.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.435+0000 7fbc277fe700 1 -- 172.21.15.82:0/2920764450 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2801f7f0 con 0x5604e8ef5860 2023-07-14T19:27:53.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- 172.21.15.82:0/2920764450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 msgr2=0x5604e8f12790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 0x5604e8f12790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800b600 tx=0x7fbc28015000 comp rx=0 tx=0).stop 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- 172.21.15.82:0/2920764450 shutdown_connections 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef6460 0x5604e8f08ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef4f10 0x5604e8f05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/2920764450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef5860 0x5604e8f12790 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- 172.21.15.82:0/2920764450 >> 172.21.15.82:0/2920764450 conn(0x5604e8eea630 msgr2=0x5604e8efd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- 172.21.15.82:0/2920764450 shutdown_connections 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- 172.21.15.82:0/2920764450 wait complete. 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- start start 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 Processor -- start 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8f0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8f0df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8f10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8f112c0 con 0x5604e8ef6460 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8f11430 con 0x5604e8ef4f10 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e8f115f0 con 0x5604e8ef5860 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.436+0000 7fbc4f109b80 10 client.0 _ll_register_callbacks cb 0x7fbc30099680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8f10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8f10d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41650/0 (socket says 172.21.15.82:41650) 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 -- 172.21.15.82:0/3068331503 learned_addr learned my addr 172.21.15.82:0/3068331503 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc3bfff700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8f0df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8f0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 -- 172.21.15.82:0/3068331503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 msgr2=0x5604e8f0df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8f0df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 -- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 msgr2=0x5604e8f0da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8f0da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 -- 172.21.15.82:0/3068331503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2800fe80 con 0x5604e8ef6460 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8f0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4150e700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8f10d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b7f0 tx=0x7fbc340093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433d4a83ccdb686 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:53.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:27:53.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3068331503 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400e070 con 0x5604e8ef6460 2023-07-14T19:27:53.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc340060c0 con 0x5604e8ef6460 2023-07-14T19:27:53.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3068331503 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fbc3401c960 con 0x5604e8ef6460 2023-07-14T19:27:53.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3068331503 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc3401cb40 con 0x5604e8ef6460 2023-07-14T19:27:53.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4f109b80 10 client.15081 fetch_fsmap learned FSMap version 22 2023-07-14T19:27:53.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5604e8c72260 con 0x5604e8ef6460 2023-07-14T19:27:53.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.437+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3068331503 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc34036020 con 0x5604e8ef6460 2023-07-14T19:27:53.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3068331503 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc34019070 con 0x5604e8ef6460 2023-07-14T19:27:53.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc4f109b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:27:53.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc4f109b80 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T19:27:53.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc4f109b80 20 client.15081 populate_metadata read hostname 'smithi082' 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f629d0 0x5604e8f5f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc4f109b80 4 client.15081 mds_command: new command op to 24524 tid=0[{"prefix": "status"}] 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5604e8e3ee60 con 0x5604e8f629d0 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f629d0 0x5604e8f5f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc1bfff700 10 client.15081 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.438+0000 7fbc40d0d700 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f629d0 0x5604e8f5f6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e99df85f608dc9 server_cookie=bda00e59379ec212 in_seq=0 out_seq=0 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc1bfff700 1 -- 172.21.15.82:0/3068331503 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5604e8e3ee60 con 0x5604e8f629d0 2023-07-14T19:27:53.449 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc1bfff700 10 client.15081 handle_command_reply: tid=0 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 client.15081 shutdown 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 msgr2=0x5604e8f10d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8f10d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b7f0 tx=0x7fbc340093b0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 shutdown_connections 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604e8ef6460 0x5604e8f10d80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604e8ef4f10 0x5604e8f0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5604e8f629d0 0x5604e8f5f6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 --2- 172.21.15.82:0/3068331503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5604e8ef5860 0x5604e8f0df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc1bfff700 0 client.15081 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:27:53.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 >> 172.21.15.82:0/3068331503 conn(0x5604e8eea630 msgr2=0x5604e8f08ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 shutdown_connections 2023-07-14T19:27:53.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 1 -- 172.21.15.82:0/3068331503 wait complete. 2023-07-14T19:27:53.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc4f109b80 20 client.15081 trim_cache size 0 max 16384 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24524, 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-07-14T19:27:53.452 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-07-14T19:27:53.453 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 55.698948057000003, 2023-07-14T19:27:53.453 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 22, 2023-07-14T19:27:53.453 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-07-14T19:27:53.453 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-07-14T19:27:53.453 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 62.539741908000003 2023-07-14T19:27:53.453 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:27:53.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbc2c0a11e0 msgr2=0x7fbc2c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbc2c0a11e0 0x7fbc2c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800bcf0 tx=0x7fbc2800b2c0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 msgr2=0x7fbc3c1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c1008c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d980 tx=0x7fbc3000dc90 comp rx=0 tx=0).stop 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 shutdown_connections 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc3c1018e0 0x7fbc3c0ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fbc2c0a11e0 0x7fbc2c0a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc3c0fdee0 0x7fbc3c1008c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.439+0000 7fbc42f71700 1 --2- 172.21.15.82:0/514282404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc3c100f50 0x7fbc3c0fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:53.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.440+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 >> 172.21.15.82:0/514282404 conn(0x7fbc3c0f98a0 msgr2=0x7fbc3c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:53.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.440+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 shutdown_connections 2023-07-14T19:27:53.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:53.440+0000 7fbc42f71700 1 -- 172.21.15.82:0/514282404 wait complete. 2023-07-14T19:27:53.456 INFO:tasks.cephadm:Setup complete, yielding 2023-07-14T19:27:53.457 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T19:27:53.467 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:27:53.467 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph orch status' 2023-07-14T19:27:53.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:53 smithi133 ceph-mon[128894]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:27:53.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:53 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3459367402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:53.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:53 smithi082 ceph-mon[127461]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:27:53.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:53 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3459367402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:53.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:53 smithi161 ceph-mon[129184]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:27:53.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:53 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3459367402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:27:54.019 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:55.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 -- 172.21.15.82:0/4173171549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700ffdb0 msgr2=0x7fba70100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:55.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 --2- 172.21.15.82:0/4173171549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700ffdb0 0x7fba70100200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba60009990 tx=0x7fba60009ca0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 -- 172.21.15.82:0/4173171549 shutdown_connections 2023-07-14T19:27:55.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 --2- 172.21.15.82:0/4173171549 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 0x7fba70107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 --2- 172.21.15.82:0/4173171549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba700febb0 0x7fba700fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 --2- 172.21.15.82:0/4173171549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700ffdb0 0x7fba70100200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 -- 172.21.15.82:0/4173171549 >> 172.21.15.82:0/4173171549 conn(0x7fba700fa1e0 msgr2=0x7fba700fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:55.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 -- 172.21.15.82:0/4173171549 shutdown_connections 2023-07-14T19:27:55.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.200+0000 7fba74ea3700 1 -- 172.21.15.82:0/4173171549 wait complete. 2023-07-14T19:27:55.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 Processor -- start 2023-07-14T19:27:55.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 -- start start 2023-07-14T19:27:55.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 0x7fba70194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:55.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba700ffdb0 0x7fba70194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:55.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 0x7fba7019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:55.204 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019c420 con 0x7fba700febb0 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019c560 con 0x7fba70100740 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba74ea3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7019c6f0 con 0x7fba700ffdb0 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba6e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 0x7fba70194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.201+0000 7fba6e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 0x7fba70194890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41658/0 (socket says 172.21.15.82:41658) 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 -- 172.21.15.82:0/1648420771 learned_addr learned my addr 172.21.15.82:0/1648420771 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6dd9b700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba700ffdb0 0x7fba70194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:55.205 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6ed9d700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 0x7fba7019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 -- 172.21.15.82:0/1648420771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba700ffdb0 msgr2=0x7fba70194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba700ffdb0 0x7fba70194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 -- 172.21.15.82:0/1648420771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 msgr2=0x7fba7019be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 0x7fba7019be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 -- 172.21.15.82:0/1648420771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600095f0 con 0x7fba700febb0 2023-07-14T19:27:55.206 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6ed9d700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 0x7fba7019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba6e59c700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 0x7fba70194890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba64009920 tx=0x7fba64009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64012070 con 0x7fba700febb0 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba70068160 con 0x7fba700febb0 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba64009e70 con 0x7fba700febb0 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64007ca0 con 0x7fba700febb0 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.202+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba70068660 con 0x7fba700febb0 2023-07-14T19:27:55.207 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.203+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7010a370 con 0x7fba700febb0 2023-07-14T19:27:55.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.204+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fba64016030 con 0x7fba700febb0 2023-07-14T19:27:55.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.205+0000 7fba5f7fe700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fba580a11e0 0x7fba580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:55.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.205+0000 7fba6dd9b700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fba580a11e0 0x7fba580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:55.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.205+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fba640cdad0 con 0x7fba700febb0 2023-07-14T19:27:55.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.205+0000 7fba6dd9b700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fba580a11e0 0x7fba580a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba6000b5c0 tx=0x7fba60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:55.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.209+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fba640971e0 con 0x7fba700febb0 2023-07-14T19:27:55.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.412+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba70060a80 con 0x7fba580a11e0 2023-07-14T19:27:55.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.413+0000 7fba5f7fe700 1 -- 172.21.15.82:0/1648420771 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fba70060a80 con 0x7fba580a11e0 2023-07-14T19:27:55.414 INFO:teuthology.orchestra.run.smithi082.stdout:Backend: cephadm 2023-07-14T19:27:55.414 INFO:teuthology.orchestra.run.smithi082.stdout:Available: Yes 2023-07-14T19:27:55.414 INFO:teuthology.orchestra.run.smithi082.stdout:Paused: No 2023-07-14T19:27:55.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fba580a11e0 msgr2=0x7fba580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:55.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fba580a11e0 0x7fba580a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba6000b5c0 tx=0x7fba60005fd0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 msgr2=0x7fba70194890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 0x7fba70194890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba64009920 tx=0x7fba64009c30 comp rx=0 tx=0).stop 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 shutdown_connections 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba70100740 0x7fba7019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fba580a11e0 0x7fba580a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba700febb0 0x7fba70194890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 --2- 172.21.15.82:0/1648420771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba700ffdb0 0x7fba70194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:55.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 >> 172.21.15.82:0/1648420771 conn(0x7fba700fa1e0 msgr2=0x7fba700fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:55.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 shutdown_connections 2023-07-14T19:27:55.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:55.415+0000 7fba74ea3700 1 -- 172.21.15.82:0/1648420771 wait complete. 2023-07-14T19:27:55.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-14T19:27:56.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:55 smithi133 ceph-mon[128894]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:56.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:55 smithi082 ceph-mon[127461]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:56.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:55 smithi161 ceph-mon[129184]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:27:56.421 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:56.955 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:56 smithi082 ceph-mon[127461]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:56 smithi133 ceph-mon[128894]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:57.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:56 smithi161 ceph-mon[129184]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:57.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 -- 172.21.15.82:0/238047526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 msgr2=0x7fb9741081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:57.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 --2- 172.21.15.82:0/238047526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741081b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00b0a0 tx=0x7fb96c00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 -- 172.21.15.82:0/238047526 shutdown_connections 2023-07-14T19:27:57.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 --2- 172.21.15.82:0/238047526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741081b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 --2- 172.21.15.82:0/238047526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9740fdf80 0x7fb9740fe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 --2- 172.21.15.82:0/238047526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb974100f60 0x7fb9741013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 -- 172.21.15.82:0/238047526 >> 172.21.15.82:0/238047526 conn(0x7fb9740f9990 msgr2=0x7fb9740fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:57.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.614+0000 7fb97a717700 1 -- 172.21.15.82:0/238047526 shutdown_connections 2023-07-14T19:27:57.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.615+0000 7fb97a717700 1 -- 172.21.15.82:0/238047526 wait complete. 2023-07-14T19:27:57.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.615+0000 7fb97a717700 1 Processor -- start 2023-07-14T19:27:57.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.615+0000 7fb97a717700 1 -- start start 2023-07-14T19:27:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb97a717700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9740fdf80 0x7fb97419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb97a717700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb974100f60 0x7fb97419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb97a717700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:57.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb97a717700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741a3cc0 con 0x7fb9741018f0 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb97a717700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741a3e00 con 0x7fb974100f60 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb97a717700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741a3f90 con 0x7fb9740fdf80 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741a36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41672/0 (socket says 172.21.15.82:41672) 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 -- 172.21.15.82:0/2200971727 learned_addr learned my addr 172.21.15.82:0/2200971727 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb973fff700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9740fdf80 0x7fb97419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:57.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb9737fe700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb974100f60 0x7fb97419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 -- 172.21.15.82:0/2200971727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9740fdf80 msgr2=0x7fb97419c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9740fdf80 0x7fb97419c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 -- 172.21.15.82:0/2200971727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb974100f60 msgr2=0x7fb97419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb974100f60 0x7fb97419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb978cb4700 1 -- 172.21.15.82:0/2200971727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c009d00 con 0x7fb9741018f0 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.616+0000 7fb973fff700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9740fdf80 0x7fb97419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:27:57.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.617+0000 7fb978cb4700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741a36f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00b070 tx=0x7fb96c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.617+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00e040 con 0x7fb9741018f0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.617+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9741a4210 con 0x7fb9741018f0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.617+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb96c003e10 con 0x7fb9741018f0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.617+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c01b7e0 con 0x7fb9741018f0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.617+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9741a4620 con 0x7fb9741018f0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.618+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb974105300 con 0x7fb9741018f0 2023-07-14T19:27:57.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.619+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb96c01da30 con 0x7fb9741018f0 2023-07-14T19:27:57.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.619+0000 7fb9717fa700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb95c0a11e0 0x7fb95c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:27:57.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.619+0000 7fb973fff700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb95c0a11e0 0x7fb95c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:27:57.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.619+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fb96c0cde00 con 0x7fb9741018f0 2023-07-14T19:27:57.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.620+0000 7fb973fff700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb95c0a11e0 0x7fb95c0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb974067f00 tx=0x7fb964009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:27:57.624 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.624+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb96c0975a0 con 0x7fb9741018f0 2023-07-14T19:27:57.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.835+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb974060a80 con 0x7fb95c0a11e0 2023-07-14T19:27:57.847 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.847+0000 7fb9717fa700 1 -- 172.21.15.82:0/2200971727 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fb974060a80 con 0x7fb95c0a11e0 2023-07-14T19:27:57.848 INFO:teuthology.orchestra.run.smithi082.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-14T19:27:57.848 INFO:teuthology.orchestra.run.smithi082.stdout:mds.a smithi082 running (69s) - 69s 16.7M - 18.0.0-4940-g81c9387d 219b180907a8 7196b46d9cff 2023-07-14T19:27:57.848 INFO:teuthology.orchestra.run.smithi082.stdout:mds.b smithi133 running (67s) - 66s 14.0M - 18.0.0-4940-g81c9387d 219b180907a8 40b3c59af0bb 2023-07-14T19:27:57.848 INFO:teuthology.orchestra.run.smithi082.stdout:mds.c smithi161 running (78s) - 77s 16.6M - 18.0.0-4940-g81c9387d 219b180907a8 b78b76947e7c 2023-07-14T19:27:57.848 INFO:teuthology.orchestra.run.smithi082.stdout:mds.d smithi082 running (84s) - 84s 14.1M - 18.0.0-4940-g81c9387d 219b180907a8 0c96277daa5e 2023-07-14T19:27:57.848 INFO:teuthology.orchestra.run.smithi082.stdout:mds.e smithi133 running (82s) - 82s 16.8M - 18.0.0-4940-g81c9387d 219b180907a8 c640fe9a4b77 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.f smithi161 running (80s) - 79s 15.8M - 18.0.0-4940-g81c9387d 219b180907a8 01792752b196 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.g smithi082 running (71s) - 71s 15.7M - 18.0.0-4940-g81c9387d 219b180907a8 f704df9e1981 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.h smithi133 running (75s) - 75s 16.5M - 18.0.0-4940-g81c9387d 219b180907a8 e6a5bcc4fb4a 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.i smithi161 running (73s) - 73s 16.8M - 18.0.0-4940-g81c9387d 219b180907a8 b63e4500df00 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.j smithi082 running (86s) - 86s 15.6M - 18.0.0-4940-g81c9387d 219b180907a8 fdd6b90b4c21 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.k smithi133 running (88s) - 88s 15.6M - 18.0.0-4940-g81c9387d 219b180907a8 6afc4397bcf6 2023-07-14T19:27:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:mds.l smithi161 running (91s) - 90s 16.8M - 18.0.0-4940-g81c9387d 219b180907a8 74519dae9ef3 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.x smithi082 *:9283,8765 running (7m) - 7m 511M - 18.0.0-4940-g81c9387d 219b180907a8 85707f902675 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.y smithi133 *:8443,8765 running (5m) - 5m 445M - 18.0.0-4940-g81c9387d 219b180907a8 1f6b6186935a 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:mgr.z smithi161 *:8443,8765 running (5m) - 5m 446M - 18.0.0-4940-g81c9387d 219b180907a8 60c177910ff1 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:mon.a smithi082 running (7m) - 7m 82.3M 2048M 18.0.0-4940-g81c9387d 219b180907a8 8c88ed0430c2 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:mon.b smithi133 running (6m) - 6m 57.8M 2048M 18.0.0-4940-g81c9387d 219b180907a8 35049331e818 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:mon.c smithi161 running (6m) - 6m 57.2M 2048M 18.0.0-4940-g81c9387d 219b180907a8 3b9c95c08a93 2023-07-14T19:27:57.850 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 smithi082 running (5m) - 5m 56.4M 4292M 18.0.0-4940-g81c9387d 219b180907a8 9ae2004bc3b3 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 smithi133 running (5m) - 5m 65.3M 4292M 18.0.0-4940-g81c9387d 219b180907a8 252d94f9f26f 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 smithi161 running (4m) - 4m 57.9M 4292M 18.0.0-4940-g81c9387d 219b180907a8 4cf3a2e24d56 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 smithi082 running (4m) - 4m 53.8M 4292M 18.0.0-4940-g81c9387d 219b180907a8 924cd1e0f01c 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 smithi133 running (4m) - 4m 49.9M 4292M 18.0.0-4940-g81c9387d 219b180907a8 c14f6ba7b784 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 smithi161 running (3m) - 3m 52.0M 4292M 18.0.0-4940-g81c9387d 219b180907a8 3cfe4d9a8f32 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 smithi082 running (3m) - 3m 47.0M 4292M 18.0.0-4940-g81c9387d 219b180907a8 1042d2456700 2023-07-14T19:27:57.851 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 smithi133 running (3m) - 3m 51.3M 4292M 18.0.0-4940-g81c9387d 219b180907a8 acc51c870348 2023-07-14T19:27:57.852 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 smithi161 running (2m) - 2m 48.3M 4292M 18.0.0-4940-g81c9387d 219b180907a8 933bd177d962 2023-07-14T19:27:57.852 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 smithi082 running (2m) - 2m 45.3M 4292M 18.0.0-4940-g81c9387d 219b180907a8 087e7affac72 2023-07-14T19:27:57.852 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 smithi133 running (2m) - 2m 49.1M 4292M 18.0.0-4940-g81c9387d 219b180907a8 16785f28693e 2023-07-14T19:27:57.852 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 smithi161 running (109s) - 108s 43.2M 4292M 18.0.0-4940-g81c9387d 219b180907a8 e3b3242b8666 2023-07-14T19:27:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb95c0a11e0 msgr2=0x7fb95c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb95c0a11e0 0x7fb95c0a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb974067f00 tx=0x7fb964009260 comp rx=0 tx=0).stop 2023-07-14T19:27:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 msgr2=0x7fb9741a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:27:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741a36f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00b070 tx=0x7fb96c00bce0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 shutdown_connections 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb95c0a11e0 0x7fb95c0a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9741018f0 0x7fb9741a36f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9740fdf80 0x7fb97419c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 --2- 172.21.15.82:0/2200971727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb974100f60 0x7fb97419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 >> 172.21.15.82:0/2200971727 conn(0x7fb9740f9990 msgr2=0x7fb974102440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 shutdown_connections 2023-07-14T19:27:57.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:27:57.849+0000 7fb97a717700 1 -- 172.21.15.82:0/2200971727 wait complete. 2023-07-14T19:27:58.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:57 smithi133 ceph-mon[128894]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T19:27:58.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:57 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:27:58.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:57 smithi082 ceph-mon[127461]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T19:27:58.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:57 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:27:58.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:57 smithi161 ceph-mon[129184]: pgmap v279: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-14T19:27:58.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:57 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:27:58.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-14T19:27:58.827 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:27:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:58 smithi133 ceph-mon[128894]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:59.203 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:58 smithi082 ceph-mon[127461]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:27:59.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:58 smithi161 ceph-mon[129184]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:00.029 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:27:59 smithi082 ceph-mon[127461]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:28:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:27:59 smithi133 ceph-mon[128894]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:28:00.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:27:59 smithi161 ceph-mon[129184]: pgmap v280: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-14T19:28:00.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 -- 172.21.15.82:0/1801956408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c0ff820 msgr2=0x7f4f4c1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:00.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 --2- 172.21.15.82:0/1801956408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c0ff820 0x7f4f4c1070f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b0d0 tx=0x7f4f4400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 -- 172.21.15.82:0/1801956408 shutdown_connections 2023-07-14T19:28:00.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 --2- 172.21.15.82:0/1801956408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c0ff820 0x7f4f4c1070f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 --2- 172.21.15.82:0/1801956408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f4c100150 0x7f4f4c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 --2- 172.21.15.82:0/1801956408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 0x7f4f4c0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 -- 172.21.15.82:0/1801956408 >> 172.21.15.82:0/1801956408 conn(0x7f4f4c0f9990 msgr2=0x7f4f4c0fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:00.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 -- 172.21.15.82:0/1801956408 shutdown_connections 2023-07-14T19:28:00.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.242+0000 7f4f524ba700 1 -- 172.21.15.82:0/1801956408 wait complete. 2023-07-14T19:28:00.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.243+0000 7f4f524ba700 1 Processor -- start 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.243+0000 7f4f524ba700 1 -- start start 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 0x7f4f4c0fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f4c0ff820 0x7f4f4c19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 0x7f4f4c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c19d600 con 0x7f4f4c100150 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c19d770 con 0x7f4f4c0fee90 2023-07-14T19:28:00.246 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c19d930 con 0x7f4f4c0ff820 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 0x7f4f4c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f4bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 0x7f4f4c0fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f4bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 0x7f4f4c0fe8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:38028/0 (socket says 172.21.15.82:38028) 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 0x7f4f4c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41684/0 (socket says 172.21.15.82:41684) 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f4bfff700 1 -- 172.21.15.82:0/4110637859 learned_addr learned my addr 172.21.15.82:0/4110637859 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f4b7fe700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f4c0ff820 0x7f4f4c19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 -- 172.21.15.82:0/4110637859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f4c0ff820 msgr2=0x7f4f4c19a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f4c0ff820 0x7f4f4c19a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 -- 172.21.15.82:0/4110637859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 msgr2=0x7f4f4c0fe8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 0x7f4f4c0fe8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 -- 172.21.15.82:0/4110637859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009d00 con 0x7f4f4c100150 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f50a57700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 0x7f4f4c19ce60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b770 tx=0x7f4f44008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400e040 con 0x7f4f4c100150 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f4401d930 con 0x7f4f4c100150 2023-07-14T19:28:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4401b930 con 0x7f4f4c100150 2023-07-14T19:28:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c19dbb0 con 0x7f4f4c100150 2023-07-14T19:28:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.244+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c19dfd0 con 0x7f4f4c100150 2023-07-14T19:28:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.245+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c106200 con 0x7f4f4c100150 2023-07-14T19:28:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.246+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4f4401d470 con 0x7f4f4c100150 2023-07-14T19:28:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.247+0000 7f4f497fa700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f340a11e0 0x7f4f340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.247+0000 7f4f4bfff700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f340a11e0 0x7f4f340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.247+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f4f440ce100 con 0x7f4f4c100150 2023-07-14T19:28:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.247+0000 7f4f4bfff700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f340a11e0 0x7f4f340a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005950 tx=0x7f4f3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.250+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4f440978a0 con 0x7f4f4c100150 2023-07-14T19:28:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.455+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f4c060a80 con 0x7f4f340a11e0 2023-07-14T19:28:00.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.460+0000 7f4f497fa700 1 -- 172.21.15.82:0/4110637859 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f4f4c060a80 con 0x7f4f340a11e0 2023-07-14T19:28:00.461 INFO:teuthology.orchestra.run.smithi082.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-14T19:28:00.461 INFO:teuthology.orchestra.run.smithi082.stdout:mds.all 12/12 - 95s smithi082=a;smithi082=d;smithi082=g;smithi082=j;smithi133=b;smithi133=e;smithi133=h;smithi133=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2023-07-14T19:28:00.461 INFO:teuthology.orchestra.run.smithi082.stdout:mgr 3/3 - 6m smithi082=x;smithi133=y;smithi161=z;count:3 2023-07-14T19:28:00.462 INFO:teuthology.orchestra.run.smithi082.stdout:mon 3/3 - 6m smithi082:172.21.15.82=a;smithi133:172.21.15.133=b;smithi161:172.21.15.161=c;count:3 2023-07-14T19:28:00.462 INFO:teuthology.orchestra.run.smithi082.stdout:osd 12 - - 2023-07-14T19:28:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.462+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f340a11e0 msgr2=0x7f4f340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.462+0000 7f4f524ba700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f340a11e0 0x7f4f340a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c005950 tx=0x7f4f3c009260 comp rx=0 tx=0).stop 2023-07-14T19:28:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.462+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 msgr2=0x7f4f4c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:00.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.462+0000 7f4f524ba700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 0x7f4f4c19ce60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b770 tx=0x7f4f44008f10 comp rx=0 tx=0).stop 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 shutdown_connections 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f4f340a11e0 0x7f4f340a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f4c100150 0x7f4f4c19ce60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c0fee90 0x7f4f4c0fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 --2- 172.21.15.82:0/4110637859 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f4c0ff820 0x7f4f4c19a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 >> 172.21.15.82:0/4110637859 conn(0x7f4f4c0f9990 msgr2=0x7f4f4c103340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 shutdown_connections 2023-07-14T19:28:00.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:00.463+0000 7f4f524ba700 1 -- 172.21.15.82:0/4110637859 wait complete. 2023-07-14T19:28:00.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-14T19:28:01.433 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:28:01.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:01.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:28:01.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:01 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:01.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:28:01.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:01 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:01.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: pgmap v281: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:28:01.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:01.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:01 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 -- 172.21.15.82:0/2921999952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600febb0 msgr2=0x7f03600fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 --2- 172.21.15.82:0/2921999952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600febb0 0x7f03600fefc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0350009990 tx=0x7f0350009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 -- 172.21.15.82:0/2921999952 shutdown_connections 2023-07-14T19:28:02.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 --2- 172.21.15.82:0/2921999952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0360100740 0x7f0360107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 --2- 172.21.15.82:0/2921999952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600febb0 0x7f03600fefc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 --2- 172.21.15.82:0/2921999952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03600ffdb0 0x7f0360100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 -- 172.21.15.82:0/2921999952 >> 172.21.15.82:0/2921999952 conn(0x7f03600fa1e0 msgr2=0x7f03600fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 -- 172.21.15.82:0/2921999952 shutdown_connections 2023-07-14T19:28:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.670+0000 7f03685e3700 1 -- 172.21.15.82:0/2921999952 wait complete. 2023-07-14T19:28:02.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 Processor -- start 2023-07-14T19:28:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 -- start start 2023-07-14T19:28:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03600febb0 0x7f0360102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 0x7f03601014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0360100740 0x7f0360101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:02.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03601021b0 con 0x7f03600ffdb0 2023-07-14T19:28:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0360102320 con 0x7f0360100740 2023-07-14T19:28:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03685e3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019fb20 con 0x7f03600febb0 2023-07-14T19:28:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 0x7f03601014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 0x7f03601014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41694/0 (socket says 172.21.15.82:41694) 2023-07-14T19:28:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 -- 172.21.15.82:0/1615603964 learned_addr learned my addr 172.21.15.82:0/1615603964 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:02.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f036637f700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03600febb0 0x7f0360102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0366b80700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0360100740 0x7f0360101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 -- 172.21.15.82:0/1615603964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03600febb0 msgr2=0x7f0360102ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03600febb0 0x7f0360102ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 -- 172.21.15.82:0/1615603964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0360100740 msgr2=0x7f0360101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0360100740 0x7f0360101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 -- 172.21.15.82:0/1615603964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500095f0 con 0x7f03600ffdb0 2023-07-14T19:28:02.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0366b80700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0360100740 0x7f0360101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:02.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f0365b7e700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 0x7f03601014f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b530 tx=0x7f035c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:02.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.671+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c011830 con 0x7f03600ffdb0 2023-07-14T19:28:02.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.672+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035c011e50 con 0x7f03600ffdb0 2023-07-14T19:28:02.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.672+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036019fe00 con 0x7f03600ffdb0 2023-07-14T19:28:02.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.672+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00f2e0 con 0x7f03600ffdb0 2023-07-14T19:28:02.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.672+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03601a02f0 con 0x7f03600ffdb0 2023-07-14T19:28:02.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.673+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0360106cf0 con 0x7f03600ffdb0 2023-07-14T19:28:02.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.674+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f035c010580 con 0x7f03600ffdb0 2023-07-14T19:28:02.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.674+0000 7f03577fe700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f034c0a11e0 0x7f034c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:02.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.674+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f035c0cc370 con 0x7f03600ffdb0 2023-07-14T19:28:02.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.674+0000 7f036637f700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f034c0a11e0 0x7f034c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:02.682 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.675+0000 7f036637f700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f034c0a11e0 0x7f034c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0350006010 tx=0x7f035000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:02.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.678+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f035c095b90 con 0x7f03600ffdb0 2023-07-14T19:28:02.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:02.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:02.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:02 smithi133 ceph-mon[128894]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:02.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:02.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:02.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:02 smithi082 ceph-mon[127461]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:02.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:02 smithi161 ceph-mon[129184]: from='mgr.14152 172.21.15.82:0/1362434968' entity='mgr.x' 2023-07-14T19:28:02.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.874+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0360060a80 con 0x7f034c0a11e0 2023-07-14T19:28:02.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.875+0000 7f03577fe700 1 -- 172.21.15.82:0/1615603964 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f0360060a80 con 0x7f034c0a11e0 2023-07-14T19:28:02.876 INFO:teuthology.orchestra.run.smithi082.stdout:HOST ADDR LABELS STATUS 2023-07-14T19:28:02.876 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 172.21.15.82 2023-07-14T19:28:02.876 INFO:teuthology.orchestra.run.smithi082.stdout:smithi133 172.21.15.133 2023-07-14T19:28:02.877 INFO:teuthology.orchestra.run.smithi082.stdout:smithi161 172.21.15.161 2023-07-14T19:28:02.877 INFO:teuthology.orchestra.run.smithi082.stdout:3 hosts in cluster 2023-07-14T19:28:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f034c0a11e0 msgr2=0x7f034c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f034c0a11e0 0x7f034c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0350006010 tx=0x7f035000b540 comp rx=0 tx=0).stop 2023-07-14T19:28:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 msgr2=0x7f03601014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 0x7f03601014f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f035c00b530 tx=0x7f035c00b8f0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 shutdown_connections 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0360100740 0x7f0360101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.877+0000 7f03685e3700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03600febb0 0x7f0360102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.878+0000 7f03685e3700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f034c0a11e0 0x7f034c0a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.878+0000 7f03685e3700 1 --2- 172.21.15.82:0/1615603964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600ffdb0 0x7f03601014f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.878+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 >> 172.21.15.82:0/1615603964 conn(0x7f03600fa1e0 msgr2=0x7f03600fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.878+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 shutdown_connections 2023-07-14T19:28:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:02.878+0000 7f03685e3700 1 -- 172.21.15.82:0/1615603964 wait complete. 2023-07-14T19:28:03.291 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-14T19:28:03.844 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:28:03.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:03 smithi133 ceph-mon[128894]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:28:03.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:03 smithi133 ceph-mon[128894]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:03.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:03 smithi082 ceph-mon[127461]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:28:03.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:03 smithi082 ceph-mon[127461]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:03.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:03 smithi161 ceph-mon[129184]: pgmap v282: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-14T19:28:03.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:03 smithi161 ceph-mon[129184]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:05.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 -- 172.21.15.82:0/3905194437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 msgr2=0x7f18bc107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:05.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 --2- 172.21.15.82:0/3905194437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc107000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f18b400b0a0 tx=0x7f18b400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 -- 172.21.15.82:0/3905194437 shutdown_connections 2023-07-14T19:28:05.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 --2- 172.21.15.82:0/3905194437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc107000 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 --2- 172.21.15.82:0/3905194437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0febb0 0x7f18bc0fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 --2- 172.21.15.82:0/3905194437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18bc0ffdb0 0x7f18bc100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 -- 172.21.15.82:0/3905194437 >> 172.21.15.82:0/3905194437 conn(0x7f18bc0fa1e0 msgr2=0x7f18bc0fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:05.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 -- 172.21.15.82:0/3905194437 shutdown_connections 2023-07-14T19:28:05.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.057+0000 7f18c0d15700 1 -- 172.21.15.82:0/3905194437 wait complete. 2023-07-14T19:28:05.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.058+0000 7f18c0d15700 1 Processor -- start 2023-07-14T19:28:05.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.058+0000 7f18c0d15700 1 -- start start 2023-07-14T19:28:05.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18bc0febb0 0x7f18bc194990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:05.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0ffdb0 0x7f18bc194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:05.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:05.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc19c520 con 0x7f18bc100740 2023-07-14T19:28:05.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc19c660 con 0x7f18bc0febb0 2023-07-14T19:28:05.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc19c7f0 con 0x7f18bc0ffdb0 2023-07-14T19:28:05.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41708/0 (socket says 172.21.15.82:41708) 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 -- 172.21.15.82:0/1104663735 learned_addr learned my addr 172.21.15.82:0/1104663735 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18b9d9b700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0ffdb0 0x7f18bc194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18ba59c700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18bc0febb0 0x7f18bc194990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 -- 172.21.15.82:0/1104663735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0ffdb0 msgr2=0x7f18bc194ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0ffdb0 0x7f18bc194ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.064 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 -- 172.21.15.82:0/1104663735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18bc0febb0 msgr2=0x7f18bc194990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:05.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18bc0febb0 0x7f18bc194990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 -- 172.21.15.82:0/1104663735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b4009d00 con 0x7f18bc100740 2023-07-14T19:28:05.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18b9d9b700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0ffdb0 0x7f18bc194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:05.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18bad9d700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc19bf50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f18b4004d00 tx=0x7f18b4004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:05.065 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b400e040 con 0x7f18bc100740 2023-07-14T19:28:05.066 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc19ca70 con 0x7f18bc100740 2023-07-14T19:28:05.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18b401bd20 con 0x7f18bc100740 2023-07-14T19:28:05.068 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4005940 con 0x7f18bc100740 2023-07-14T19:28:05.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.059+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc19cf30 con 0x7f18bc100740 2023-07-14T19:28:05.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.060+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc106150 con 0x7f18bc100740 2023-07-14T19:28:05.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.061+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f18b4005040 con 0x7f18bc100740 2023-07-14T19:28:05.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.062+0000 7f18b37fe700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f18a80a11e0 0x7f18a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:05.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.062+0000 7f18ba59c700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f18a80a11e0 0x7f18a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:05.069 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.062+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f18b40cead0 con 0x7f18bc100740 2023-07-14T19:28:05.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.062+0000 7f18ba59c700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f18a80a11e0 0x7f18a80a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18a4005fd0 tx=0x7f18a4005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:05.070 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.065+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f18b4098270 con 0x7f18bc100740 2023-07-14T19:28:05.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.261+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f18bc060a80 con 0x7f18a80a11e0 2023-07-14T19:28:05.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.263+0000 7f18b37fe700 1 -- 172.21.15.82:0/1104663735 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f18bc060a80 con 0x7f18a80a11e0 2023-07-14T19:28:05.264 INFO:teuthology.orchestra.run.smithi082.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-14T19:28:05.264 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 2m ago LVM detected, locked 2023-07-14T19:28:05.264 INFO:teuthology.orchestra.run.smithi082.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 2m ago LVM detected, locked 2023-07-14T19:28:05.264 INFO:teuthology.orchestra.run.smithi082.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 110s ago LVM detected, locked 2023-07-14T19:28:05.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f18a80a11e0 msgr2=0x7f18a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:05.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f18a80a11e0 0x7f18a80a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f18a4005fd0 tx=0x7f18a4005dd0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 msgr2=0x7f18bc19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:05.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc19bf50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f18b4004d00 tx=0x7f18b4004de0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 shutdown_connections 2023-07-14T19:28:05.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f18a80a11e0 0x7f18a80a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18bc100740 0x7f18bc19bf50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f18bc0febb0 0x7f18bc194990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 --2- 172.21.15.82:0/1104663735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f18bc0ffdb0 0x7f18bc194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:05.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 >> 172.21.15.82:0/1104663735 conn(0x7f18bc0fa1e0 msgr2=0x7f18bc103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:05.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 shutdown_connections 2023-07-14T19:28:05.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:05.265+0000 7f18c0d15700 1 -- 172.21.15.82:0/1104663735 wait complete. 2023-07-14T19:28:05.698 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-14T19:28:05.709 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:28:05.710 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-14T19:28:05.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:05 smithi082 ceph-mon[127461]: pgmap v283: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:28:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:05 smithi133 ceph-mon[128894]: pgmap v283: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:28:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:05 smithi161 ceph-mon[129184]: pgmap v283: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:28:06.263 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:28:07.107 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:06 smithi082 ceph-mon[127461]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:07.107 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:06 smithi082 ceph-mon[127461]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-14T19:28:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:06 smithi133 ceph-mon[128894]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:07.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:06 smithi133 ceph-mon[128894]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-14T19:28:07.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:06 smithi161 ceph-mon[129184]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:07.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:06 smithi161 ceph-mon[129184]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-07-14T19:28:07.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 -- 172.21.15.82:0/4228389291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e81058c0 msgr2=0x7f96e8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:07.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 --2- 172.21.15.82:0/4228389291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e81058c0 0x7f96e8107d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96e000b0a0 tx=0x7f96e000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 -- 172.21.15.82:0/4228389291 shutdown_connections 2023-07-14T19:28:07.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 --2- 172.21.15.82:0/4228389291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e81058c0 0x7f96e8107d70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 --2- 172.21.15.82:0/4228389291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96e8069750 0x7f96e8102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 --2- 172.21.15.82:0/4228389291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e8102db0 0x7f96e8105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 -- 172.21.15.82:0/4228389291 >> 172.21.15.82:0/4228389291 conn(0x7f96e80fa240 msgr2=0x7f96e80fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:07.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 -- 172.21.15.82:0/4228389291 shutdown_connections 2023-07-14T19:28:07.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.564+0000 7f96edbdf700 1 -- 172.21.15.82:0/4228389291 wait complete. 2023-07-14T19:28:07.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 Processor -- start 2023-07-14T19:28:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 -- start start 2023-07-14T19:28:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e8102db0 0x7f96e8195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96e81058c0 0x7f96e8195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 0x7f96e819d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819d960 con 0x7f96e819afb0 2023-07-14T19:28:07.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819daa0 con 0x7f96e8102db0 2023-07-14T19:28:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96edbdf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e819dc30 con 0x7f96e81058c0 2023-07-14T19:28:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 0x7f96e819d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96e7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 0x7f96e819d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41722/0 (socket says 172.21.15.82:41722) 2023-07-14T19:28:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96e7fff700 1 -- 172.21.15.82:0/3322673758 learned_addr learned my addr 172.21.15.82:0/3322673758 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96e6ffd700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96e81058c0 0x7f96e8195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:07.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.565+0000 7f96e77fe700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e8102db0 0x7f96e8195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e7fff700 1 -- 172.21.15.82:0/3322673758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96e81058c0 msgr2=0x7f96e8195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e7fff700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96e81058c0 0x7f96e8195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e7fff700 1 -- 172.21.15.82:0/3322673758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e8102db0 msgr2=0x7f96e8195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e7fff700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e8102db0 0x7f96e8195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e7fff700 1 -- 172.21.15.82:0/3322673758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0009d00 con 0x7f96e819afb0 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e7fff700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 0x7f96e819d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96e0004d00 tx=0x7f96e0004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000e040 con 0x7f96e819afb0 2023-07-14T19:28:07.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e819deb0 con 0x7f96e819afb0 2023-07-14T19:28:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96e001bd20 con 0x7f96e819afb0 2023-07-14T19:28:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0005940 con 0x7f96e819afb0 2023-07-14T19:28:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.566+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e819e2c0 con 0x7f96e819afb0 2023-07-14T19:28:07.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.567+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e80fbdb0 con 0x7f96e819afb0 2023-07-14T19:28:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.569+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f96e0005040 con 0x7f96e819afb0 2023-07-14T19:28:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.569+0000 7f96e4ff9700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f96d40a1150 0x7f96d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:07.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.569+0000 7f96e77fe700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f96d40a1150 0x7f96d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.569+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f96e00ce9a0 con 0x7f96e819afb0 2023-07-14T19:28:07.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.570+0000 7f96e77fe700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f96d40a1150 0x7f96d40a3600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96e8069480 tx=0x7f96d8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:07.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.573+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f96e0098240 con 0x7f96e819afb0 2023-07-14T19:28:07.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.799+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f96e802cba0 con 0x7f96e819afb0 2023-07-14T19:28:07.800 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.799+0000 7f96e4ff9700 1 -- 172.21.15.82:0/3322673758 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2893 (secure 0 0 0) 0x7f96e009b860 con 0x7f96e819afb0 2023-07-14T19:28:07.800 INFO:teuthology.orchestra.run.smithi082.stdout:e22 2023-07-14T19:28:07.801 INFO:teuthology.orchestra.run.smithi082.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-14T19:28:07.801 INFO:teuthology.orchestra.run.smithi082.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T19:28:07.801 INFO:teuthology.orchestra.run.smithi082.stdout:legacy client fscid: 1 2023-07-14T19:28:07.801 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:07.801 INFO:teuthology.orchestra.run.smithi082.stdout:Filesystem 'cephfs' (1) 2023-07-14T19:28:07.801 INFO:teuthology.orchestra.run.smithi082.stdout:fs_name cephfs 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 22 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:created 2023-07-14T19:26:56.908603+0000 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2023-07-14T19:27:02.519960+0000 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:tableserver 0 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:root 0 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:session_timeout 300 2023-07-14T19:28:07.802 INFO:teuthology.orchestra.run.smithi082.stdout:session_autoclose 300 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:max_file_size 1099511627776 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:max_xattr_size 65536 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:required_client_features {} 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure 0 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:last_failure_osd_epoch 0 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:max_mds 3 2023-07-14T19:28:07.803 INFO:teuthology.orchestra.run.smithi082.stdout:in 0,1,2 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:up {0=24524,1=24491,2=24481} 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:failed 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:damaged 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:stopped 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:data_pools [3] 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:metadata_pool 2 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:inline_data disabled 2023-07-14T19:28:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:balancer 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:bal_rank_mask -1 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:standby_count_wanted 0 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b{0:24524} state up:active seq 3 addr [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.161:6836/42384414,v1:172.21.15.161:6837/42384414] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.h{1:24491} state up:active seq 6 addr [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.e{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.133:6834/3278917095,v1:172.21.15.133:6835/3278917095] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.805 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.i{2:24481} state up:active seq 6 addr [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.f{2:24451} state up:standby-replay seq 1 addr [v2:172.21.15.161:6834/3902735752,v1:172.21.15.161:6835/3902735752] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout:Standby daemons: 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.82:6834/1071311199,v1:172.21.15.82:6835/1071311199] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.82:6836/464864339,v1:172.21.15.82:6837/464864339] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.806 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.g{-1:14622} state up:standby seq 1 addr [v2:172.21.15.82:6838/3224571334,v1:172.21.15.82:6839/3224571334] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.807 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a{-1:14634} state up:standby seq 1 addr [v2:172.21.15.82:6840/2495420363,v1:172.21.15.82:6841/2495420363] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.807 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.161:6832/2795796645,v1:172.21.15.161:6833/2795796645] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.807 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.k{-1:24431} state up:standby seq 1 addr [v2:172.21.15.133:6832/1264023541,v1:172.21.15.133:6833/1264023541] compat {c=[1],r=[1],i=[7ff]}] 2023-07-14T19:28:07.807 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.801+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f96d40a1150 msgr2=0x7f96d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.801+0000 7f96edbdf700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f96d40a1150 0x7f96d40a3600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96e8069480 tx=0x7f96d8005d50 comp rx=0 tx=0).stop 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.801+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 msgr2=0x7f96e819d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.801+0000 7f96edbdf700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 0x7f96e819d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96e0004d00 tx=0x7f96e0004de0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 shutdown_connections 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f96d40a1150 0x7f96d40a3600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96e819afb0 0x7f96e819d390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.808 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96e8102db0 0x7f96e8195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 --2- 172.21.15.82:0/3322673758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96e81058c0 0x7f96e8195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:07.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 >> 172.21.15.82:0/3322673758 conn(0x7f96e80fa240 msgr2=0x7f96e81010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:07.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 shutdown_connections 2023-07-14T19:28:07.809 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:07.802+0000 7f96edbdf700 1 -- 172.21.15.82:0/3322673758 wait complete. 2023-07-14T19:28:07.809 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:28:08.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:07 smithi133 ceph-mon[128894]: pgmap v284: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T19:28:08.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:07 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3322673758' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T19:28:08.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:07 smithi082 ceph-mon[127461]: pgmap v284: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T19:28:08.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:07 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3322673758' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T19:28:08.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-14T19:28:08.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:07 smithi161 ceph-mon[129184]: pgmap v284: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T19:28:08.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:07 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3322673758' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-14T19:28:08.771 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:28:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 -- 172.21.15.82:0/1919486991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a4100240 msgr2=0x7fa4a4100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/1919486991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a4100240 0x7fa4a4100650 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa498009990 tx=0x7fa498009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 -- 172.21.15.82:0/1919486991 shutdown_connections 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/1919486991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a4107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/1919486991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a4100240 0x7fa4a4100650 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/1919486991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4a40fee10 0x7fa4a40ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 -- 172.21.15.82:0/1919486991 >> 172.21.15.82:0/1919486991 conn(0x7fa4a40f9990 msgr2=0x7fa4a40fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 -- 172.21.15.82:0/1919486991 shutdown_connections 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.980+0000 7fa4a8e69700 1 -- 172.21.15.82:0/1919486991 wait complete. 2023-07-14T19:28:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 Processor -- start 2023-07-14T19:28:09.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 -- start start 2023-07-14T19:28:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40fee10 0x7fa4a4102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a41014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4a4100240 0x7fa4a41019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4102170 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41022e0 con 0x7fa4a40fee10 2023-07-14T19:28:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a8e69700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a419fa30 con 0x7fa4a4100240 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a41014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a41014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41734/0 (socket says 172.21.15.82:41734) 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 -- 172.21.15.82:0/3662007954 learned_addr learned my addr 172.21.15.82:0/3662007954 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a259c700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40fee10 0x7fa4a4102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a2d9d700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4a4100240 0x7fa4a41019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 -- 172.21.15.82:0/3662007954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4a4100240 msgr2=0x7fa4a41019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4a4100240 0x7fa4a41019f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 -- 172.21.15.82:0/3662007954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40fee10 msgr2=0x7fa4a4102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40fee10 0x7fa4a4102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.981+0000 7fa4a1d9b700 1 -- 172.21.15.82:0/3662007954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980095f0 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4a259c700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40fee10 0x7fa4a4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4a1d9b700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a41014b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c920 tx=0x7fa49400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494007b00 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa494004cf0 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a419fd10 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494005610 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.982+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a41a01c0 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.983+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a4106cb0 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.984+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa494004750 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.984+0000 7fa4937fe700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa48c0a1150 0x7fa48c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.984+0000 7fa4a259c700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa48c0a1150 0x7fa48c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.984+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fa4940cc850 con 0x7fa4a40ff7c0 2023-07-14T19:28:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.985+0000 7fa4a259c700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa48c0a1150 0x7fa48c0a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa498006010 tx=0x7fa49800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:09.989+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa494096060 con 0x7fa4a40ff7c0 2023-07-14T19:28:10.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.175+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fa4a402c8e0 con 0x7fa4a40ff7c0 2023-07-14T19:28:10.176 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.176+0000 7fa4937fe700 1 -- 172.21.15.82:0/3662007954 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) v1 ==== 56+0+5208 (secure 0 0 0) 0x7fa49401d070 con 0x7fa4a40ff7c0 2023-07-14T19:28:10.176 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 83 2023-07-14T19:28:10.176 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:28:10.176 INFO:teuthology.orchestra.run.smithi082.stdout:created 2023-07-14T19:20:12.915999+0000 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:modified 2023-07-14T19:27:05.776753+0000 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:crush_version 26 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:full_ratio 0.95 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:backfillfull_ratio 0.9 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:nearfull_ratio 0.85 2023-07-14T19:28:10.177 INFO:teuthology.orchestra.run.smithi082.stdout:require_min_compat_client luminous 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:min_compat_client luminous 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:require_osd_release reef 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:stretch_mode_enabled false 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:max_osd 12 2023-07-14T19:28:10.178 INFO:teuthology.orchestra.run.smithi082.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6802/2396885961,v1:172.21.15.82:6803/2396885961] [v2:172.21.15.82:6804/2396885961,v1:172.21.15.82:6805/2396885961] exists,up ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6800/1413395016,v1:172.21.15.133:6801/1413395016] [v2:172.21.15.133:6802/1413395016,v1:172.21.15.133:6803/1413395016] exists,up 087df2cd-4b6e-4ca5-a4ac-13083d804ef4 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/4228439619,v1:172.21.15.161:6801/4228439619] [v2:172.21.15.161:6802/4228439619,v1:172.21.15.161:6803/4228439619] exists,up 359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.3 up in weight 1 up_from 26 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6810/595190701,v1:172.21.15.82:6811/595190701] [v2:172.21.15.82:6812/595190701,v1:172.21.15.82:6813/595190701] exists,up aabce84f-f9f4-45cc-b738-7414623bae2c 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.4 up in weight 1 up_from 33 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6808/3870116585,v1:172.21.15.133:6809/3870116585] [v2:172.21.15.133:6810/3870116585,v1:172.21.15.133:6811/3870116585] exists,up f44de9db-95a0-4007-8acb-40ac3ed8161e 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.5 up in weight 1 up_from 38 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/4025083410,v1:172.21.15.161:6809/4025083410] [v2:172.21.15.161:6810/4025083410,v1:172.21.15.161:6811/4025083410] exists,up 351f3277-08c4-46e3-b1ae-1f78290f556a 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.6 up in weight 1 up_from 45 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6818/2376027432,v1:172.21.15.82:6819/2376027432] [v2:172.21.15.82:6820/2376027432,v1:172.21.15.82:6821/2376027432] exists,up 69302d50-5480-44c9-815d-c580832c5198 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.7 up in weight 1 up_from 50 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6816/3133253156,v1:172.21.15.133:6817/3133253156] [v2:172.21.15.133:6818/3133253156,v1:172.21.15.133:6819/3133253156] exists,up a4fa7360-353c-455e-a2d5-4fe907ce87ab 2023-07-14T19:28:10.179 INFO:teuthology.orchestra.run.smithi082.stdout:osd.8 up in weight 1 up_from 57 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/119023360,v1:172.21.15.161:6817/119023360] [v2:172.21.15.161:6818/119023360,v1:172.21.15.161:6819/119023360] exists,up 16b227e3-291b-4c62-943f-9262f1114940 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:osd.9 up in weight 1 up_from 64 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6826/3744771950,v1:172.21.15.82:6827/3744771950] [v2:172.21.15.82:6828/3744771950,v1:172.21.15.82:6829/3744771950] exists,up 300cc59c-8947-4ec0-9352-784783f12ff2 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:osd.10 up in weight 1 up_from 69 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6824/2098424958,v1:172.21.15.133:6825/2098424958] [v2:172.21.15.133:6826/2098424958,v1:172.21.15.133:6827/2098424958] exists,up 13ec79ce-2a2b-4930-83e8-98530ae6fe96 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:osd.11 up in weight 1 up_from 76 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/2831052737,v1:172.21.15.161:6825/2831052737] [v2:172.21.15.161:6826/2831052737,v1:172.21.15.161:6827/2831052737] exists,up 9803dc37-4330-4d09-b2db-38f8fac795c8 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.a [2,11] 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 2.13 [2,11] 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.11 [5,9] 2023-07-14T19:28:10.180 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.2c [4,9] 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:pg_upmap_items 3.34 [4,9] 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/2410218140 expires 2023-07-15T19:21:04.644444+0000 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/1257676906 expires 2023-07-15T19:21:04.644444+0000 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/1035467551 expires 2023-07-15T19:21:04.644444+0000 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/1035467551 expires 2023-07-15T19:21:04.644444+0000 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6801/3383156270 expires 2023-07-15T19:20:42.182360+0000 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/998146971 expires 2023-07-15T19:20:42.182360+0000 2023-07-14T19:28:10.181 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:6800/3383156270 expires 2023-07-15T19:20:42.182360+0000 2023-07-14T19:28:10.182 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/690674119 expires 2023-07-15T19:20:42.182360+0000 2023-07-14T19:28:10.182 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/3474653214 expires 2023-07-15T19:21:04.644444+0000 2023-07-14T19:28:10.182 INFO:teuthology.orchestra.run.smithi082.stdout:blocklist 172.21.15.82:0/10878106 expires 2023-07-15T19:20:42.182360+0000 2023-07-14T19:28:10.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa48c0a1150 msgr2=0x7fa48c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa48c0a1150 0x7fa48c0a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa498006010 tx=0x7fa49800b540 comp rx=0 tx=0).stop 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 msgr2=0x7fa4a41014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a41014b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa49400c920 tx=0x7fa49400cce0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 shutdown_connections 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4a4100240 0x7fa4a41019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4a40fee10 0x7fa4a4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fa48c0a1150 0x7fa48c0a3600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 --2- 172.21.15.82:0/3662007954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4a40ff7c0 0x7fa4a41014b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 >> 172.21.15.82:0/3662007954 conn(0x7fa4a40f9990 msgr2=0x7fa4a40fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:10.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 shutdown_connections 2023-07-14T19:28:10.184 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.178+0000 7fa4a8e69700 1 -- 172.21.15.82:0/3662007954 wait complete. 2023-07-14T19:28:10.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:09 smithi133 ceph-mon[128894]: pgmap v285: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:10.201 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:09 smithi082 ceph-mon[127461]: pgmap v285: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:10.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:09 smithi161 ceph-mon[129184]: pgmap v285: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:10.611 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-14T19:28:10.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:10.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 -- 172.21.15.82:0/939650015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 msgr2=0x7f97b0106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 --2- 172.21.15.82:0/939650015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b0106f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f97a800b0a0 tx=0x7f97a800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 -- 172.21.15.82:0/939650015 shutdown_connections 2023-07-14T19:28:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 --2- 172.21.15.82:0/939650015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b0106f70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 --2- 172.21.15.82:0/939650015 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b00feb30 0x7f97b00fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 --2- 172.21.15.82:0/939650015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97b00ffd20 0x7f97b0100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 -- 172.21.15.82:0/939650015 >> 172.21.15.82:0/939650015 conn(0x7f97b00fa110 msgr2=0x7f97b00fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 -- 172.21.15.82:0/939650015 shutdown_connections 2023-07-14T19:28:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.749+0000 7f97b7c41700 1 -- 172.21.15.82:0/939650015 wait complete. 2023-07-14T19:28:10.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 Processor -- start 2023-07-14T19:28:10.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 -- start start 2023-07-14T19:28:10.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97b00feb30 0x7f97b01959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b00ffd20 0x7f97b0195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b019d540 con 0x7f97b01006b0 2023-07-14T19:28:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b019d680 con 0x7f97b00feb30 2023-07-14T19:28:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b7c41700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b019d810 con 0x7f97b00ffd20 2023-07-14T19:28:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41750/0 (socket says 172.21.15.82:41750) 2023-07-14T19:28:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 -- 172.21.15.82:0/1195324180 learned_addr learned my addr 172.21.15.82:0/1195324180 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b59dd700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97b00feb30 0x7f97b01959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 -- 172.21.15.82:0/1195324180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b00ffd20 msgr2=0x7f97b0195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b00ffd20 0x7f97b0195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 -- 172.21.15.82:0/1195324180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97b00feb30 msgr2=0x7f97b01959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.750+0000 7f97b61de700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97b00feb30 0x7f97b01959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97b61de700 1 -- 172.21.15.82:0/1195324180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8009d00 con 0x7f97b01006b0 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97b61de700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b019cf70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f97a800b830 tx=0x7f97a8004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a800e040 con 0x7f97b01006b0 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97a80094c0 con 0x7f97b01006b0 2023-07-14T19:28:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b019da90 con 0x7f97b01006b0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a801bc70 con 0x7f97b01006b0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.751+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b019df10 con 0x7f97b01006b0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.752+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b0105700 con 0x7f97b01006b0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.753+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f97a801d890 con 0x7f97b01006b0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.753+0000 7f97a6ffd700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f979c0a11a0 0x7f979c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.753+0000 7f97b59dd700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f979c0a11a0 0x7f979c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.753+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f97a80ce9a0 con 0x7f97b01006b0 2023-07-14T19:28:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.754+0000 7f97b59dd700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f979c0a11a0 0x7f979c0a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97a0005950 tx=0x7f97a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:10.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.756+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f97a80982c0 con 0x7f97b01006b0 2023-07-14T19:28:10.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.949+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97b002c9d0 con 0x7f97b01006b0 2023-07-14T19:28:10.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.950+0000 7f97a6ffd700 1 -- 172.21.15.82:0/1195324180 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f97a8021090 con 0x7f97b01006b0 2023-07-14T19:28:10.951 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:10.952 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f979c0a11a0 msgr2=0x7f979c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f979c0a11a0 0x7f979c0a3650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97a0005950 tx=0x7f97a00058e0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 msgr2=0x7f97b019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b019cf70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f97a800b830 tx=0x7f97a8004d30 comp rx=0 tx=0).stop 2023-07-14T19:28:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 shutdown_connections 2023-07-14T19:28:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f979c0a11a0 0x7f979c0a3650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97b01006b0 0x7f97b019cf70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97b00feb30 0x7f97b01959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 --2- 172.21.15.82:0/1195324180 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97b00ffd20 0x7f97b0195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 >> 172.21.15.82:0/1195324180 conn(0x7f97b00fa110 msgr2=0x7f97b00fbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 shutdown_connections 2023-07-14T19:28:10.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:10.952+0000 7f97b7c41700 1 -- 172.21.15.82:0/1195324180 wait complete. 2023-07-14T19:28:10.965 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:28:10.966 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:28:11.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 -- 172.21.15.82:0/3338098226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 msgr2=0x7f79fc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/3338098226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc106f60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f79f400b0a0 tx=0x7f79f400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 -- 172.21.15.82:0/3338098226 shutdown_connections 2023-07-14T19:28:11.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/3338098226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc106f60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/3338098226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79fc0feb20 0x7f79fc0fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/3338098226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79fc0ffd10 0x7f79fc100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 -- 172.21.15.82:0/3338098226 >> 172.21.15.82:0/3338098226 conn(0x7f79fc0fa160 msgr2=0x7f79fc0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:11.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 -- 172.21.15.82:0/3338098226 shutdown_connections 2023-07-14T19:28:11.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.090+0000 7f7a02cc1700 1 -- 172.21.15.82:0/3338098226 wait complete. 2023-07-14T19:28:11.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 Processor -- start 2023-07-14T19:28:11.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 -- start start 2023-07-14T19:28:11.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79fc0feb20 0x7f79fc069320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79fc0ffd10 0x7f79fc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc19d550 con 0x7f79fc1006a0 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc19d6c0 con 0x7f79fc0ffd10 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a02cc1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc19d880 con 0x7f79fc0feb20 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a0125e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a0125e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41764/0 (socket says 172.21.15.82:41764) 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a0125e700 1 -- 172.21.15.82:0/2170995238 learned_addr learned my addr 172.21.15.82:0/2170995238 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f79fbfff700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79fc0ffd10 0x7f79fc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.091+0000 7f7a00a5d700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79fc0feb20 0x7f79fc069320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a0125e700 1 -- 172.21.15.82:0/2170995238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79fc0feb20 msgr2=0x7f79fc069320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a0125e700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79fc0feb20 0x7f79fc069320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a0125e700 1 -- 172.21.15.82:0/2170995238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79fc0ffd10 msgr2=0x7f79fc19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a0125e700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79fc0ffd10 0x7f79fc19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a0125e700 1 -- 172.21.15.82:0/2170995238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4009d00 con 0x7f79fc1006a0 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a0125e700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc19cdb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79f400b780 tx=0x7f79f400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f400e040 con 0x7f79fc1006a0 2023-07-14T19:28:11.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79f4004e60 con 0x7f79fc1006a0 2023-07-14T19:28:11.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc19db00 con 0x7f79fc1006a0 2023-07-14T19:28:11.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f401b870 con 0x7f79fc1006a0 2023-07-14T19:28:11.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.092+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc19df80 con 0x7f79fc1006a0 2023-07-14T19:28:11.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.093+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc106010 con 0x7f79fc1006a0 2023-07-14T19:28:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.094+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f79f401b9d0 con 0x7f79fc1006a0 2023-07-14T19:28:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.095+0000 7f79f9ffb700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f79ec0a11d0 0x7f79ec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.095+0000 7f7a00a5d700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f79ec0a11d0 0x7f79ec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.095+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f79f40ce160 con 0x7f79fc1006a0 2023-07-14T19:28:11.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.095+0000 7f7a00a5d700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f79ec0a11d0 0x7f79ec0a3680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f79f0005fd0 tx=0x7f79f0005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:11.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.099+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f79f4097960 con 0x7f79fc1006a0 2023-07-14T19:28:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:10 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3662007954' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T19:28:11.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:10 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3662007954' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T19:28:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:10 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3662007954' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-14T19:28:11.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.290+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79fc061250 con 0x7f79fc1006a0 2023-07-14T19:28:11.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.290+0000 7f79f9ffb700 1 -- 172.21.15.82:0/2170995238 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79f401bc80 con 0x7f79fc1006a0 2023-07-14T19:28:11.291 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:28:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f79ec0a11d0 msgr2=0x7f79ec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f79ec0a11d0 0x7f79ec0a3680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f79f0005fd0 tx=0x7f79f0005dd0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 msgr2=0x7f79fc19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc19cdb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79f400b780 tx=0x7f79f400bdb0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 shutdown_connections 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f79ec0a11d0 0x7f79ec0a3680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79fc1006a0 0x7f79fc19cdb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79fc0feb20 0x7f79fc069320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 --2- 172.21.15.82:0/2170995238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79fc0ffd10 0x7f79fc19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 >> 172.21.15.82:0/2170995238 conn(0x7f79fc0fa160 msgr2=0x7f79fc103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 shutdown_connections 2023-07-14T19:28:11.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.292+0000 7f7a02cc1700 1 -- 172.21.15.82:0/2170995238 wait complete. 2023-07-14T19:28:11.305 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:28:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 -- 172.21.15.82:0/2977063165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac0fed80 msgr2=0x7f45ac0ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 --2- 172.21.15.82:0/2977063165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac0fed80 0x7f45ac0ff1f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f459c009990 tx=0x7f459c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 -- 172.21.15.82:0/2977063165 shutdown_connections 2023-07-14T19:28:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 --2- 172.21.15.82:0/2977063165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ff730 0x7f45ac106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 --2- 172.21.15.82:0/2977063165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac1001c0 0x7f45ac1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 --2- 172.21.15.82:0/2977063165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac0fed80 0x7f45ac0ff1f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 -- 172.21.15.82:0/2977063165 >> 172.21.15.82:0/2977063165 conn(0x7f45ac0f98e0 msgr2=0x7f45ac0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 -- 172.21.15.82:0/2977063165 shutdown_connections 2023-07-14T19:28:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 -- 172.21.15.82:0/2977063165 wait complete. 2023-07-14T19:28:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.428+0000 7f45b18bc700 1 Processor -- start 2023-07-14T19:28:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 -- start start 2023-07-14T19:28:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0fed80 0x7f45ac102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ff730 0x7f45ac101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 0x7f45ac1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac101e50 con 0x7f45ac1001c0 2023-07-14T19:28:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac101fc0 con 0x7f45ac0ff730 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b18bc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19f9c0 con 0x7f45ac0fed80 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 0x7f45ac1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 0x7f45ac1016b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41778/0 (socket says 172.21.15.82:41778) 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 -- 172.21.15.82:0/3544873892 learned_addr learned my addr 172.21.15.82:0/3544873892 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45aaffd700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0fed80 0x7f45ac102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45aa7fc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ff730 0x7f45ac101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 -- 172.21.15.82:0/3544873892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0fed80 msgr2=0x7f45ac102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0fed80 0x7f45ac102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 -- 172.21.15.82:0/3544873892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ff730 msgr2=0x7f45ac101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ff730 0x7f45ac101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 -- 172.21.15.82:0/3544873892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0095f0 con 0x7f45ac1001c0 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45aaffd700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0fed80 0x7f45ac102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45ab7fe700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 0x7f45ac1016b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45a4007ee0 tx=0x7f45a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.429+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4017070 con 0x7f45ac1001c0 2023-07-14T19:28:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.430+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45a400f040 con 0x7f45ac1001c0 2023-07-14T19:28:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.430+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac19fca0 con 0x7f45ac1001c0 2023-07-14T19:28:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.430+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4005740 con 0x7f45ac1001c0 2023-07-14T19:28:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.430+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac1a0160 con 0x7f45ac1001c0 2023-07-14T19:28:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.431+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac10a360 con 0x7f45ac1001c0 2023-07-14T19:28:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.432+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f45a40058a0 con 0x7f45ac1001c0 2023-07-14T19:28:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.432+0000 7f45b08ba700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f45940a1150 0x7f45940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.432+0000 7f45aaffd700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f45940a1150 0x7f45940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.432+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f45a40cc0e0 con 0x7f45ac1001c0 2023-07-14T19:28:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.433+0000 7f45aaffd700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f45940a1150 0x7f45940a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005950 tx=0x7f45a0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.436+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f45a40958f0 con 0x7f45ac1001c0 2023-07-14T19:28:11.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.626+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45ac1a0710 con 0x7f45ac1001c0 2023-07-14T19:28:11.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.627+0000 7f45b08ba700 1 -- 172.21.15.82:0/3544873892 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45a4098f10 con 0x7f45ac1001c0 2023-07-14T19:28:11.631 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:28:11.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f45940a1150 msgr2=0x7f45940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f45940a1150 0x7f45940a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005950 tx=0x7f45a0005f50 comp rx=0 tx=0).stop 2023-07-14T19:28:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 msgr2=0x7f45ac1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 0x7f45ac1016b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45a4007ee0 tx=0x7f45a400d3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 shutdown_connections 2023-07-14T19:28:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f45940a1150 0x7f45940a3600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45ac1001c0 0x7f45ac1016b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45ac0fed80 0x7f45ac102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 --2- 172.21.15.82:0/3544873892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac0ff730 0x7f45ac101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 >> 172.21.15.82:0/3544873892 conn(0x7f45ac0f98e0 msgr2=0x7f45ac0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:11.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 shutdown_connections 2023-07-14T19:28:11.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.629+0000 7f45b18bc700 1 -- 172.21.15.82:0/3544873892 wait complete. 2023-07-14T19:28:11.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:28:11.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 -- 172.21.15.82:0/1952886282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550103010 msgr2=0x7f85501053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 --2- 172.21.15.82:0/1952886282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550103010 0x7f85501053f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f853c009990 tx=0x7f853c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 -- 172.21.15.82:0/1952886282 shutdown_connections 2023-07-14T19:28:11.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 --2- 172.21.15.82:0/1952886282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550105930 0x7f8550107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 --2- 172.21.15.82:0/1952886282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85501006f0 0x7f8550102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 --2- 172.21.15.82:0/1952886282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550103010 0x7f85501053f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 -- 172.21.15.82:0/1952886282 >> 172.21.15.82:0/1952886282 conn(0x7f85500fa130 msgr2=0x7f85500fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 -- 172.21.15.82:0/1952886282 shutdown_connections 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.766+0000 7f8556ed4700 1 -- 172.21.15.82:0/1952886282 wait complete. 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 Processor -- start 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 -- start start 2023-07-14T19:28:11.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85501006f0 0x7f8550195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550103010 0x7f8550195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 0x7f855019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855019d510 con 0x7f8550105930 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855019d650 con 0x7f8550103010 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8556ed4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855019d7e0 con 0x7f85501006f0 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8555471700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 0x7f855019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8555471700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 0x7f855019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41792/0 (socket says 172.21.15.82:41792) 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f8555471700 1 -- 172.21.15.82:0/4266600509 learned_addr learned my addr 172.21.15.82:0/4266600509 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.767+0000 7f854ffff700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550103010 0x7f8550195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8554c70700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85501006f0 0x7f8550195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8555471700 1 -- 172.21.15.82:0/4266600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85501006f0 msgr2=0x7f8550195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8555471700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85501006f0 0x7f8550195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8555471700 1 -- 172.21.15.82:0/4266600509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550103010 msgr2=0x7f8550195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8555471700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550103010 0x7f8550195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8555471700 1 -- 172.21.15.82:0/4266600509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c0095f0 con 0x7f8550105930 2023-07-14T19:28:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f854ffff700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550103010 0x7f8550195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8555471700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 0x7f855019cf40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8548007ee0 tx=0x7f854800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548017070 con 0x7f8550105930 2023-07-14T19:28:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854800f040 con 0x7f8550105930 2023-07-14T19:28:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548005740 con 0x7f8550105930 2023-07-14T19:28:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855019dac0 con 0x7f8550105930 2023-07-14T19:28:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.768+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855019df80 con 0x7f8550105930 2023-07-14T19:28:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.769+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855010b420 con 0x7f8550105930 2023-07-14T19:28:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.770+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f85480058a0 con 0x7f8550105930 2023-07-14T19:28:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.771+0000 7f854dffb700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f85400a11d0 0x7f85400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.771+0000 7f8554c70700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f85400a11d0 0x7f85400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.771+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f85480cc6f0 con 0x7f8550105930 2023-07-14T19:28:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.771+0000 7f8554c70700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f85400a11d0 0x7f85400a3680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8544005950 tx=0x7f85440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:11.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.774+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8548095f00 con 0x7f8550105930 2023-07-14T19:28:11.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.962+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f855019e590 con 0x7f8550105930 2023-07-14T19:28:11.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.962+0000 7f854dffb700 1 -- 172.21.15.82:0/4266600509 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8548099520 con 0x7f8550105930 2023-07-14T19:28:11.963 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:28:11.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:11 smithi082 ceph-mon[127461]: pgmap v286: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:11.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:11 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1195324180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:11.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:11 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2170995238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:11.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:11 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3544873892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f85400a11d0 msgr2=0x7f85400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f85400a11d0 0x7f85400a3680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8544005950 tx=0x7f85440058e0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 msgr2=0x7f855019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 0x7f855019cf40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8548007ee0 tx=0x7f854800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 shutdown_connections 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f85400a11d0 0x7f85400a3680 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8550105930 0x7f855019cf40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f85501006f0 0x7f8550195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 --2- 172.21.15.82:0/4266600509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8550103010 0x7f8550195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 >> 172.21.15.82:0/4266600509 conn(0x7f85500fa130 msgr2=0x7f8550101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 shutdown_connections 2023-07-14T19:28:11.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:11.964+0000 7f8556ed4700 1 -- 172.21.15.82:0/4266600509 wait complete. 2023-07-14T19:28:11.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:28:12.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 -- 172.21.15.82:0/3580417132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0100730 msgr2=0x7fc0a0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 --2- 172.21.15.82:0/3580417132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0100730 0x7fc0a0102b10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc094009990 tx=0x7fc094009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 -- 172.21.15.82:0/3580417132 shutdown_connections 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 --2- 172.21.15.82:0/3580417132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0a0105970 0x7fc0a0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 --2- 172.21.15.82:0/3580417132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0100730 0x7fc0a0102b10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 --2- 172.21.15.82:0/3580417132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a0103050 0x7fc0a0105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 -- 172.21.15.82:0/3580417132 >> 172.21.15.82:0/3580417132 conn(0x7fc0a00fa190 msgr2=0x7fc0a00fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.098+0000 7fc0a554a700 1 -- 172.21.15.82:0/3580417132 shutdown_connections 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 -- 172.21.15.82:0/3580417132 wait complete. 2023-07-14T19:28:12.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 Processor -- start 2023-07-14T19:28:12.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 -- start start 2023-07-14T19:28:12.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a0100730 0x7fc0a01958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 0x7fc0a0195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0a0105970 0x7fc0a019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019d460 con 0x7fc0a0103050 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019d5a0 con 0x7fc0a0105970 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.099+0000 7fc0a554a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a019d730 con 0x7fc0a0100730 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 0x7fc0a0195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 0x7fc0a0195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41804/0 (socket says 172.21.15.82:41804) 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 -- 172.21.15.82:0/2279389816 learned_addr learned my addr 172.21.15.82:0/2279389816 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09effd700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a0100730 0x7fc0a01958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09f7fe700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0a0105970 0x7fc0a019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 -- 172.21.15.82:0/2279389816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a0100730 msgr2=0x7fc0a01958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a0100730 0x7fc0a01958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 -- 172.21.15.82:0/2279389816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0a0105970 msgr2=0x7fc0a019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0a0105970 0x7fc0a019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 -- 172.21.15.82:0/2279389816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940095f0 con 0x7fc0a0103050 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc09e7fc700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 0x7fc0a0195e20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc09000d950 tx=0x7fc09000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000f890 con 0x7fc0a0103050 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a019da10 con 0x7fc0a0103050 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc090010460 con 0x7fc0a0103050 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000e610 con 0x7fc0a0103050 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.100+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a019df70 con 0x7fc0a0103050 2023-07-14T19:28:12.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.101+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a00fbd00 con 0x7fc0a0103050 2023-07-14T19:28:12.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.102+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc0900108d0 con 0x7fc0a0103050 2023-07-14T19:28:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.103+0000 7fc087fff700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc0880a11a0 0x7fc0880a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.103+0000 7fc09effd700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc0880a11a0 0x7fc0880a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.103+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fc0900cc5a0 con 0x7fc0a0103050 2023-07-14T19:28:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.103+0000 7fc09effd700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc0880a11a0 0x7fc0880a3650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc09400b5c0 tx=0x7fc094005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.106+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc090095db0 con 0x7fc0a0103050 2023-07-14T19:28:12.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:11 smithi133 ceph-mon[128894]: pgmap v286: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:12.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:11 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1195324180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:11 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2170995238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:12.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:11 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3544873892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:12.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:11 smithi161 ceph-mon[129184]: pgmap v286: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:12.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:11 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1195324180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:12.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:11 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2170995238' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:12.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:11 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3544873892' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:12.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.341+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0a002c8e0 con 0x7fc0a0103050 2023-07-14T19:28:12.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.341+0000 7fc087fff700 1 -- 172.21.15.82:0/2279389816 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10823 (secure 0 0 0) 0x7fc090016090 con 0x7fc0a0103050 2023-07-14T19:28:12.343 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:12.343 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:27:02.519960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:28:12.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc0880a11a0 msgr2=0x7fc0880a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc0880a11a0 0x7fc0880a3650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc09400b5c0 tx=0x7fc094005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 msgr2=0x7fc0a0195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 0x7fc0a0195e20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc09000d950 tx=0x7fc09000dc60 comp rx=0 tx=0).stop 2023-07-14T19:28:12.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 shutdown_connections 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0a0105970 0x7fc0a019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0a0100730 0x7fc0a01958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fc0880a11a0 0x7fc0880a3650 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 --2- 172.21.15.82:0/2279389816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0a0103050 0x7fc0a0195e20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.343+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 >> 172.21.15.82:0/2279389816 conn(0x7fc0a00fa190 msgr2=0x7fc0a01013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.344+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 shutdown_connections 2023-07-14T19:28:12.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.344+0000 7fc0a554a700 1 -- 172.21.15.82:0/2279389816 wait complete. 2023-07-14T19:28:12.347 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:28:12.357 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:12.478 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 -- 172.21.15.82:0/2029438360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 msgr2=0x7f3a0c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/2029438360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c100530 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009990 tx=0x7f3a00009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 -- 172.21.15.82:0/2029438360 shutdown_connections 2023-07-14T19:28:12.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/2029438360 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 0x7f3a0c106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/2029438360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c100530 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.479 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/2029438360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a0c0fed70 0x7f3a0c0ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 -- 172.21.15.82:0/2029438360 >> 172.21.15.82:0/2029438360 conn(0x7f3a0c0f98f0 msgr2=0x7f3a0c0fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:12.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 -- 172.21.15.82:0/2029438360 shutdown_connections 2023-07-14T19:28:12.480 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.477+0000 7f3a10cf8700 1 -- 172.21.15.82:0/2029438360 wait complete. 2023-07-14T19:28:12.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 Processor -- start 2023-07-14T19:28:12.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 -- start start 2023-07-14T19:28:12.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a0c0fed70 0x7f3a0c0fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 0x7f3a0c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c19d3e0 con 0x7f3a0c100120 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c19d550 con 0x7f3a0c0fed70 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a10cf8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c19d710 con 0x7f3a0c0ff720 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a09d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 0x7f3a0c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41818/0 (socket says 172.21.15.82:41818) 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a09d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 0x7f3a0c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:51080/0 (socket says 172.21.15.82:51080) 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 -- 172.21.15.82:0/701597117 learned_addr learned my addr 172.21.15.82:0/701597117 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0a59c700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a0c0fed70 0x7f3a0c0fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 -- 172.21.15.82:0/701597117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 msgr2=0x7f3a0c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 0x7f3a0c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 -- 172.21.15.82:0/701597117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a0c0fed70 msgr2=0x7f3a0c0fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a0c0fed70 0x7f3a0c0fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.478+0000 7f3a0ad9d700 1 -- 172.21.15.82:0/701597117 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a000095f0 con 0x7f3a0c100120 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.479+0000 7f3a0ad9d700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c19cc10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3a04007ee0 tx=0x7f3a0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.479+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04017070 con 0x7f3a0c100120 2023-07-14T19:28:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.479+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a0400f040 con 0x7f3a0c100120 2023-07-14T19:28:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.479+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c19d9f0 con 0x7f3a0c100120 2023-07-14T19:28:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.479+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04005580 con 0x7f3a0c100120 2023-07-14T19:28:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.479+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c19deb0 con 0x7f3a0c100120 2023-07-14T19:28:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.480+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c105ac0 con 0x7f3a0c100120 2023-07-14T19:28:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.481+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3a0400db20 con 0x7f3a0c100120 2023-07-14T19:28:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.481+0000 7f39fb7fe700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f39f40a11a0 0x7f39f40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.481+0000 7f3a0a59c700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f39f40a11a0 0x7f39f40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.482+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f3a040cc610 con 0x7f3a0c100120 2023-07-14T19:28:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.482+0000 7f3a0a59c700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f39f40a11a0 0x7f39f40a3650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:12.487 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.485+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3a04095e20 con 0x7f3a0c100120 2023-07-14T19:28:12.679 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.678+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a0c02cb30 con 0x7f3a0c100120 2023-07-14T19:28:12.680 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.679+0000 7f39fb7fe700 1 -- 172.21.15.82:0/701597117 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f3a04099440 con 0x7f3a0c100120 2023-07-14T19:28:12.681 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:12.681 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f39f40a11a0 msgr2=0x7f39f40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f39f40a11a0 0x7f39f40a3650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a000058e0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 msgr2=0x7f3a0c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c19cc10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3a04007ee0 tx=0x7f3a0400d3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 shutdown_connections 2023-07-14T19:28:12.683 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f39f40a11a0 0x7f39f40a3650 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a0c100120 0x7f3a0c19cc10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a0c0fed70 0x7f3a0c0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 --2- 172.21.15.82:0/701597117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a0c0ff720 0x7f3a0c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 >> 172.21.15.82:0/701597117 conn(0x7f3a0c0f98f0 msgr2=0x7f3a0c0fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 shutdown_connections 2023-07-14T19:28:12.684 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.681+0000 7f3a10cf8700 1 -- 172.21.15.82:0/701597117 wait complete. 2023-07-14T19:28:12.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:28:12.696 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:28:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 -- 172.21.15.82:0/2443418712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900feb10 msgr2=0x7f91900fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.826 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 --2- 172.21.15.82:0/2443418712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900feb10 0x7f91900fef20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9180009990 tx=0x7f9180009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 -- 172.21.15.82:0/2443418712 shutdown_connections 2023-07-14T19:28:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 --2- 172.21.15.82:0/2443418712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9190100690 0x7f9190106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 --2- 172.21.15.82:0/2443418712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900feb10 0x7f91900fef20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 --2- 172.21.15.82:0/2443418712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91900ffd00 0x7f9190100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 -- 172.21.15.82:0/2443418712 >> 172.21.15.82:0/2443418712 conn(0x7f91900fa130 msgr2=0x7f91900fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:12.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 -- 172.21.15.82:0/2443418712 shutdown_connections 2023-07-14T19:28:12.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.820+0000 7f91951dc700 1 -- 172.21.15.82:0/2443418712 wait complete. 2023-07-14T19:28:12.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 Processor -- start 2023-07-14T19:28:12.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 -- start start 2023-07-14T19:28:12.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91900feb10 0x7f9190194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 0x7f9190194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9190100690 0x7f919019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919019c4a0 con 0x7f91900ffd00 2023-07-14T19:28:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919019c5e0 con 0x7f91900feb10 2023-07-14T19:28:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f91951dc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919019c770 con 0x7f9190100690 2023-07-14T19:28:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f918e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 0x7f9190194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f918e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 0x7f9190194e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41830/0 (socket says 172.21.15.82:41830) 2023-07-14T19:28:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f918e59c700 1 -- 172.21.15.82:0/2347823526 learned_addr learned my addr 172.21.15.82:0/2347823526 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f918ed9d700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91900feb10 0x7f9190194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.821+0000 7f918f59e700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9190100690 0x7f919019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918e59c700 1 -- 172.21.15.82:0/2347823526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9190100690 msgr2=0x7f919019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918e59c700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9190100690 0x7f919019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918e59c700 1 -- 172.21.15.82:0/2347823526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91900feb10 msgr2=0x7f9190194920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918e59c700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91900feb10 0x7f9190194920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918e59c700 1 -- 172.21.15.82:0/2347823526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184009520 con 0x7f91900ffd00 2023-07-14T19:28:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918ed9d700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91900feb10 0x7f9190194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f918e59c700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 0x7f9190194e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f918400eb30 tx=0x7f918400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184009880 con 0x7f91900ffd00 2023-07-14T19:28:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91840044e0 con 0x7f91900ffd00 2023-07-14T19:28:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91800095f0 con 0x7f91900ffd00 2023-07-14T19:28:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184003e90 con 0x7f91900ffd00 2023-07-14T19:28:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.822+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9190067c40 con 0x7f91900ffd00 2023-07-14T19:28:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.823+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9190068690 con 0x7f91900ffd00 2023-07-14T19:28:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.824+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9184004000 con 0x7f91900ffd00 2023-07-14T19:28:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.825+0000 7f9177fff700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f91780a1150 0x7f91780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.825+0000 7f918ed9d700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f91780a1150 0x7f91780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:12.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.825+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f9184014070 con 0x7f91900ffd00 2023-07-14T19:28:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.825+0000 7f918ed9d700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f91780a1150 0x7f91780a3600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f9180005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:12.829+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f91840964b0 con 0x7f91900ffd00 2023-07-14T19:28:12.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:12 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4266600509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:12.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:12 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2279389816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:12.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:12 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/701597117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:13.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.020+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f919019eb40 con 0x7f91900ffd00 2023-07-14T19:28:13.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.020+0000 7f9177fff700 1 -- 172.21.15.82:0/2347823526 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9184099ad0 con 0x7f91900ffd00 2023-07-14T19:28:13.022 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:28:13.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.022+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f91780a1150 msgr2=0x7f91780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.022+0000 7f91951dc700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f91780a1150 0x7f91780a3600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f9180005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.022+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 msgr2=0x7f9190194e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.022+0000 7f91951dc700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 0x7f9190194e60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f918400eb30 tx=0x7f918400ee40 comp rx=0 tx=0).stop 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 shutdown_connections 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9190100690 0x7f919019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91900feb10 0x7f9190194920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f91780a1150 0x7f91780a3600 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 --2- 172.21.15.82:0/2347823526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91900ffd00 0x7f9190194e60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 >> 172.21.15.82:0/2347823526 conn(0x7f91900fa130 msgr2=0x7f91900fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 shutdown_connections 2023-07-14T19:28:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.023+0000 7f91951dc700 1 -- 172.21.15.82:0/2347823526 wait complete. 2023-07-14T19:28:13.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:28:13.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.159+0000 7f64a279d700 1 -- 172.21.15.82:0/883849921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0ffd00 msgr2=0x7f649c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.159+0000 7f64a279d700 1 --2- 172.21.15.82:0/883849921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0ffd00 0x7f649c100150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6490009990 tx=0x7f6490009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- 172.21.15.82:0/883849921 shutdown_connections 2023-07-14T19:28:13.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 --2- 172.21.15.82:0/883849921 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c100690 0x7f649c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 --2- 172.21.15.82:0/883849921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f649c0feb10 0x7f649c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 --2- 172.21.15.82:0/883849921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0ffd00 0x7f649c100150 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- 172.21.15.82:0/883849921 >> 172.21.15.82:0/883849921 conn(0x7f649c0fa130 msgr2=0x7f649c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- 172.21.15.82:0/883849921 shutdown_connections 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- 172.21.15.82:0/883849921 wait complete. 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 Processor -- start 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- start start 2023-07-14T19:28:13.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 0x7f649c194920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c0ffd00 0x7f649c194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f649c100690 0x7f649c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c19c4a0 con 0x7f649c0feb10 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c19c5e0 con 0x7f649c100690 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.160+0000 7f64a279d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c19c770 con 0x7f649c0ffd00 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 0x7f649c194920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 0x7f649c194920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41842/0 (socket says 172.21.15.82:41842) 2023-07-14T19:28:13.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 -- 172.21.15.82:0/1183763969 learned_addr learned my addr 172.21.15.82:0/1183763969 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649b7fe700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c0ffd00 0x7f649c194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 -- 172.21.15.82:0/1183763969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c0ffd00 msgr2=0x7f649c194e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c0ffd00 0x7f649c194e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 -- 172.21.15.82:0/1183763969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f649c100690 msgr2=0x7f649c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f649c100690 0x7f649c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 -- 172.21.15.82:0/1183763969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900095f0 con 0x7f649c0feb10 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649b7fe700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c0ffd00 0x7f649c194e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:13.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f649bfff700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 0x7f649c194920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d980 tx=0x7f648c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00de60 con 0x7f649c0feb10 2023-07-14T19:28:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648c0044e0 con 0x7f649c0feb10 2023-07-14T19:28:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c19ca50 con 0x7f649c0feb10 2023-07-14T19:28:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c003be0 con 0x7f649c0feb10 2023-07-14T19:28:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.161+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c19cf60 con 0x7f649c0feb10 2023-07-14T19:28:13.168 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.162+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c1060a0 con 0x7f649c0feb10 2023-07-14T19:28:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.164+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f648c009bb0 con 0x7f649c0feb10 2023-07-14T19:28:13.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.164+0000 7f64997fa700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f64840a1150 0x7f64840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.165+0000 7f649b7fe700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f64840a1150 0x7f64840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.165+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f648c0cca30 con 0x7f649c0feb10 2023-07-14T19:28:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.165+0000 7f649b7fe700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f64840a1150 0x7f64840a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6490006010 tx=0x7f649000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:13.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.168+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f648c0961a0 con 0x7f649c0feb10 2023-07-14T19:28:13.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:12 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4266600509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:12 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2279389816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:13.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:12 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/701597117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:13.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:12 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4266600509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:13.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:12 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2279389816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:13.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:12 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/701597117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.359+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f649c02cb50 con 0x7f649c0feb10 2023-07-14T19:28:13.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.359+0000 7f64997fa700 1 -- 172.21.15.82:0/1183763969 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f648c009940 con 0x7f649c0feb10 2023-07-14T19:28:13.361 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:28:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.361+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f64840a1150 msgr2=0x7f64840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.361+0000 7f64a279d700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f64840a1150 0x7f64840a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6490006010 tx=0x7f649000b540 comp rx=0 tx=0).stop 2023-07-14T19:28:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.361+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 msgr2=0x7f649c194920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.361+0000 7f64a279d700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 0x7f649c194920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f648c00d980 tx=0x7f648c00dd40 comp rx=0 tx=0).stop 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 shutdown_connections 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f649c100690 0x7f649c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f64840a1150 0x7f64840a3600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0feb10 0x7f649c194920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 --2- 172.21.15.82:0/1183763969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f649c0ffd00 0x7f649c194e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 >> 172.21.15.82:0/1183763969 conn(0x7f649c0fa130 msgr2=0x7f649c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 shutdown_connections 2023-07-14T19:28:13.365 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.362+0000 7f64a279d700 1 -- 172.21.15.82:0/1183763969 wait complete. 2023-07-14T19:28:13.375 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:28:13.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 -- 172.21.15.82:0/2652110775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100fee00 msgr2=0x7f89100ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 --2- 172.21.15.82:0/2652110775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100fee00 0x7f89100ff270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8900009990 tx=0x7f8900009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 -- 172.21.15.82:0/2652110775 shutdown_connections 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 --2- 172.21.15.82:0/2652110775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89100ff7b0 0x7f8910107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 --2- 172.21.15.82:0/2652110775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8910100240 0x7f8910100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 --2- 172.21.15.82:0/2652110775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100fee00 0x7f89100ff270 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 -- 172.21.15.82:0/2652110775 >> 172.21.15.82:0/2652110775 conn(0x7f89100f9980 msgr2=0x7f89100fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 -- 172.21.15.82:0/2652110775 shutdown_connections 2023-07-14T19:28:13.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.498+0000 7f8915a31700 1 -- 172.21.15.82:0/2652110775 wait complete. 2023-07-14T19:28:13.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 Processor -- start 2023-07-14T19:28:13.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 -- start start 2023-07-14T19:28:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89100fee00 0x7f8910195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89100ff7b0 0x7f8910195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 0x7f891019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891019d4e0 con 0x7f8910100240 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891019d620 con 0x7f89100ff7b0 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f8915a31700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891019d7b0 con 0x7f89100fee00 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f890f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 0x7f891019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f890f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 0x7f891019cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41860/0 (socket says 172.21.15.82:41860) 2023-07-14T19:28:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.499+0000 7f890f7fe700 1 -- 172.21.15.82:0/1245808390 learned_addr learned my addr 172.21.15.82:0/1245808390 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890effd700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89100fee00 0x7f8910195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890e7fc700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89100ff7b0 0x7f8910195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890f7fe700 1 -- 172.21.15.82:0/1245808390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89100fee00 msgr2=0x7f8910195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890f7fe700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89100fee00 0x7f8910195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890f7fe700 1 -- 172.21.15.82:0/1245808390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89100ff7b0 msgr2=0x7f8910195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890f7fe700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89100ff7b0 0x7f8910195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890f7fe700 1 -- 172.21.15.82:0/1245808390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000095f0 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f890f7fe700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 0x7f891019cf10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f890800ef30 tx=0x7f890800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908018070 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f890800f040 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891019da30 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908004020 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.500+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891019df40 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.501+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910106d00 con 0x7f8910100240 2023-07-14T19:28:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.502+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8908004ad0 con 0x7f8910100240 2023-07-14T19:28:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.503+0000 7f8914a2f700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f88f80a1150 0x7f88f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.503+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f89080cc450 con 0x7f8910100240 2023-07-14T19:28:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.503+0000 7f890effd700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f88f80a1150 0x7f88f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.503+0000 7f890effd700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f88f80a1150 0x7f88f80a3600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8904009bf0 tx=0x7f8904009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.507+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8908095cf0 con 0x7f8910100240 2023-07-14T19:28:13.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.695+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f891002caf0 con 0x7f8910100240 2023-07-14T19:28:13.697 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.695+0000 7f8914a2f700 1 -- 172.21.15.82:0/1245808390 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8908099310 con 0x7f8910100240 2023-07-14T19:28:13.697 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:28:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f88f80a1150 msgr2=0x7f88f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f88f80a1150 0x7f88f80a3600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8904009bf0 tx=0x7f8904009260 comp rx=0 tx=0).stop 2023-07-14T19:28:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 msgr2=0x7f891019cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 0x7f891019cf10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f890800ef30 tx=0x7f890800c450 comp rx=0 tx=0).stop 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 shutdown_connections 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f88f80a1150 0x7f88f80a3600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910100240 0x7f891019cf10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89100fee00 0x7f8910195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 --2- 172.21.15.82:0/1245808390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89100ff7b0 0x7f8910195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.697+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 >> 172.21.15.82:0/1245808390 conn(0x7f89100f9980 msgr2=0x7f89100fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.698+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 shutdown_connections 2023-07-14T19:28:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.698+0000 7f8915a31700 1 -- 172.21.15.82:0/1245808390 wait complete. 2023-07-14T19:28:13.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:28:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 -- 172.21.15.82:0/1223993495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 msgr2=0x7ff2700fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 --2- 172.21.15.82:0/1223993495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff2700fef10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff260009990 tx=0x7ff260009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 -- 172.21.15.82:0/1223993495 shutdown_connections 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 --2- 172.21.15.82:0/1223993495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff270100680 0x7ff270106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 --2- 172.21.15.82:0/1223993495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff2700fef10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 --2- 172.21.15.82:0/1223993495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2700ffcf0 0x7ff270100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 -- 172.21.15.82:0/1223993495 >> 172.21.15.82:0/1223993495 conn(0x7ff2700fa100 msgr2=0x7ff2700fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 -- 172.21.15.82:0/1223993495 shutdown_connections 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 -- 172.21.15.82:0/1223993495 wait complete. 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.835+0000 7ff2756d6700 1 Processor -- start 2023-07-14T19:28:13.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 -- start start 2023-07-14T19:28:13.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff270068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.840 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2700ffcf0 0x7ff27019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff270100680 0x7ff27019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019f670 con 0x7ff2700feb00 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019f7e0 con 0x7ff2700ffcf0 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff2756d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019f9a0 con 0x7ff270100680 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff270068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff270068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41870/0 (socket says 172.21.15.82:41870) 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 -- 172.21.15.82:0/4028853883 learned_addr learned my addr 172.21.15.82:0/4028853883 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:13.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26e7fc700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2700ffcf0 0x7ff27019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26f7fe700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff270100680 0x7ff27019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 -- 172.21.15.82:0/4028853883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff270100680 msgr2=0x7ff27019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff270100680 0x7ff27019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 -- 172.21.15.82:0/4028853883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2700ffcf0 msgr2=0x7ff27019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2700ffcf0 0x7ff27019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 -- 172.21.15.82:0/4028853883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600095f0 con 0x7ff2700feb00 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26e7fc700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2700ffcf0 0x7ff27019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:13.842 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.836+0000 7ff26effd700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff270068330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff26000b620 tx=0x7ff260005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.837+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26001d070 con 0x7ff2700feb00 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.837+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27019fc20 con 0x7ff2700feb00 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.837+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2600047e0 con 0x7ff2700feb00 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.837+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff260005560 con 0x7ff2700feb00 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.837+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2701a00a0 con 0x7ff2700feb00 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.838+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff270106090 con 0x7ff2700feb00 2023-07-14T19:28:13.843 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.839+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff260004950 con 0x7ff2700feb00 2023-07-14T19:28:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.839+0000 7ff257fff700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff2580a11a0 0x7ff2580a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.839+0000 7ff26e7fc700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff2580a11a0 0x7ff2580a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:13.844 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.840+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7ff2600cdde0 con 0x7ff2700feb00 2023-07-14T19:28:13.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.840+0000 7ff26e7fc700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff2580a11a0 0x7ff2580a3650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff2700680a0 tx=0x7ff264009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:13.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:13.843+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff2600975e0 con 0x7ff2700feb00 2023-07-14T19:28:13.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:13 smithi082 ceph-mon[127461]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-14T19:28:13.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:13 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2347823526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:13.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:13 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1183763969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:13.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:13 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1245808390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.076+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff27002c8e0 con 0x7ff2700feb00 2023-07-14T19:28:14.079 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.076+0000 7ff257fff700 1 -- 172.21.15.82:0/4028853883 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10823 (secure 0 0 0) 0x7ff26009ac00 con 0x7ff2700feb00 2023-07-14T19:28:14.080 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:14.080 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:27:02.519960+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:28:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.078+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff2580a11a0 msgr2=0x7ff2580a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.078+0000 7ff2756d6700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff2580a11a0 0x7ff2580a3650 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff2700680a0 tx=0x7ff264009260 comp rx=0 tx=0).stop 2023-07-14T19:28:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.078+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 msgr2=0x7ff270068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.078+0000 7ff2756d6700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff270068330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff26000b620 tx=0x7ff260005ef0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 shutdown_connections 2023-07-14T19:28:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff270100680 0x7ff27019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff2580a11a0 0x7ff2580a3650 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2700feb00 0x7ff270068330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 --2- 172.21.15.82:0/4028853883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2700ffcf0 0x7ff27019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 >> 172.21.15.82:0/4028853883 conn(0x7ff2700fa100 msgr2=0x7ff2701031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 shutdown_connections 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.079+0000 7ff2756d6700 1 -- 172.21.15.82:0/4028853883 wait complete. 2023-07-14T19:28:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 22 2023-07-14T19:28:14.092 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:13 smithi133 ceph-mon[128894]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-14T19:28:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:13 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2347823526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:13 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1183763969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:13 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1245808390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 -- 172.21.15.82:0/838714978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc0ffd00 msgr2=0x7f8abc100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/838714978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc0ffd00 0x7f8abc100150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009990 tx=0x7f8aa4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 -- 172.21.15.82:0/838714978 shutdown_connections 2023-07-14T19:28:14.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/838714978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8abc100690 0x7f8abc106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/838714978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8abc0feb10 0x7f8abc0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/838714978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc0ffd00 0x7f8abc100150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 -- 172.21.15.82:0/838714978 >> 172.21.15.82:0/838714978 conn(0x7f8abc0fa130 msgr2=0x7f8abc0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 -- 172.21.15.82:0/838714978 shutdown_connections 2023-07-14T19:28:14.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.212+0000 7f8ac08d4700 1 -- 172.21.15.82:0/838714978 wait complete. 2023-07-14T19:28:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 Processor -- start 2023-07-14T19:28:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 -- start start 2023-07-14T19:28:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8abc0feb10 0x7f8abc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8abc0ffd00 0x7f8abc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 0x7f8abc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc19d3e0 con 0x7f8abc100690 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc19d550 con 0x7f8abc0feb10 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ac08d4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc19d710 con 0x7f8abc0ffd00 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 0x7f8abc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 0x7f8abc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41888/0 (socket says 172.21.15.82:41888) 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 -- 172.21.15.82:0/3490599342 learned_addr learned my addr 172.21.15.82:0/3490599342 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8aba59c700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8abc0feb10 0x7f8abc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8ab9d9b700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8abc0ffd00 0x7f8abc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 -- 172.21.15.82:0/3490599342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8abc0ffd00 msgr2=0x7f8abc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8abc0ffd00 0x7f8abc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 -- 172.21.15.82:0/3490599342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8abc0feb10 msgr2=0x7f8abc069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8abc0feb10 0x7f8abc069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.213+0000 7f8abad9d700 1 -- 172.21.15.82:0/3490599342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa40095f0 con 0x7f8abc100690 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.214+0000 7f8abad9d700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 0x7f8abc19cc40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400ef30 tx=0x7f8ab400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.214+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4018070 con 0x7f8abc100690 2023-07-14T19:28:14.219 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.214+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc19d990 con 0x7f8abc100690 2023-07-14T19:28:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.214+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab400f040 con 0x7f8abc100690 2023-07-14T19:28:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.214+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4003dc0 con 0x7f8abc100690 2023-07-14T19:28:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.214+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc19de50 con 0x7f8abc100690 2023-07-14T19:28:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.215+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc105950 con 0x7f8abc100690 2023-07-14T19:28:14.220 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.216+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8ab400caa0 con 0x7f8abc100690 2023-07-14T19:28:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.216+0000 7f8ab37fe700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8aa80a11a0 0x7f8aa80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.216+0000 7f8aba59c700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8aa80a11a0 0x7f8aa80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.216+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f8ab40ccea0 con 0x7f8abc100690 2023-07-14T19:28:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.217+0000 7f8aba59c700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8aa80a11a0 0x7f8aa80a3650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005fd0 tx=0x7f8aac005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.220+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8ab4096740 con 0x7f8abc100690 2023-07-14T19:28:14.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:13 smithi161 ceph-mon[129184]: pgmap v287: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-07-14T19:28:14.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:13 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2347823526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:13 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1183763969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:13 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1245808390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:14.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.411+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8abc02c8e0 con 0x7f8abc100690 2023-07-14T19:28:14.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.413+0000 7f8ab37fe700 1 -- 172.21.15.82:0/3490599342 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f8ab4099d60 con 0x7f8abc100690 2023-07-14T19:28:14.415 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:14.415 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:14.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8aa80a11a0 msgr2=0x7f8aa80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8aa80a11a0 0x7f8aa80a3650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8aac005fd0 tx=0x7f8aac005e50 comp rx=0 tx=0).stop 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 msgr2=0x7f8abc19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 0x7f8abc19cc40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400ef30 tx=0x7f8ab400c450 comp rx=0 tx=0).stop 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 shutdown_connections 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8aa80a11a0 0x7f8aa80a3650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8abc100690 0x7f8abc19cc40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8abc0feb10 0x7f8abc069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 --2- 172.21.15.82:0/3490599342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8abc0ffd00 0x7f8abc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 >> 172.21.15.82:0/3490599342 conn(0x7f8abc0fa130 msgr2=0x7f8abc0fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 shutdown_connections 2023-07-14T19:28:14.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.415+0000 7f8ac08d4700 1 -- 172.21.15.82:0/3490599342 wait complete. 2023-07-14T19:28:14.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-14T19:26:56.908603+0000', 'modified': '2023-07-14T19:27:02.519960+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24491, 'mds_2': 24481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6835/3902735752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 3902735752}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 3902735752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/3278917095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3278917095}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 3278917095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6837/42384414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 42384414}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 42384414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.161:6839/4007435956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 4007435956}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 4007435956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24491': {'gid': 24491, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6837/46070579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 46070579}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 46070579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6839/242970631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 242970631}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 242970631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-14T19:28:14.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-14T19:28:14.430 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-14T19:28:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 -- 172.21.15.82:0/3487861567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 msgr2=0x7f3bd80fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 --2- 172.21.15.82:0/3487861567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd80fdcf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009990 tx=0x7f3bc8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 -- 172.21.15.82:0/3487861567 shutdown_connections 2023-07-14T19:28:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 --2- 172.21.15.82:0/3487861567 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bd80fe310 0x7f3bd80fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 --2- 172.21.15.82:0/3487861567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 0x7f3bd8101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 --2- 172.21.15.82:0/3487861567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd80fdcf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 -- 172.21.15.82:0/3487861567 >> 172.21.15.82:0/3487861567 conn(0x7f3bd80f98a0 msgr2=0x7f3bd80fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 -- 172.21.15.82:0/3487861567 shutdown_connections 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 -- 172.21.15.82:0/3487861567 wait complete. 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 Processor -- start 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 -- start start 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bd80fe310 0x7f3bd8195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.551+0000 7f3bdf793700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 0x7f3bd8195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdf793700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdf793700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819d4f0 con 0x7f3bd8101bd0 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdf793700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819d630 con 0x7f3bd8101280 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdf793700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd819d7c0 con 0x7f3bd80fe310 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41902/0 (socket says 172.21.15.82:41902) 2023-07-14T19:28:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 -- 172.21.15.82:0/365386959 learned_addr learned my addr 172.21.15.82:0/365386959 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 0x7f3bd8195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bd80fe310 0x7f3bd8195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 -- 172.21.15.82:0/365386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bd80fe310 msgr2=0x7f3bd8195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bd80fe310 0x7f3bd8195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 -- 172.21.15.82:0/365386959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 msgr2=0x7f3bd8195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 0x7f3bd8195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 -- 172.21.15.82:0/365386959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc80095f0 con 0x7f3bd8101bd0 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 0x7f3bd8195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bddd30700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd819cf20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0007ee0 tx=0x7f3bd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bce7fc700 1 -- 172.21.15.82:0/365386959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000ddd0 con 0x7f3bd8101bd0 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bce7fc700 1 -- 172.21.15.82:0/365386959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bd000f040 con 0x7f3bd8101bd0 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd819daa0 con 0x7f3bd8101bd0 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bce7fc700 1 -- 172.21.15.82:0/365386959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0015400 con 0x7f3bd8101bd0 2023-07-14T19:28:14.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.552+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd819dfb0 con 0x7f3bd8101bd0 2023-07-14T19:28:14.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.554+0000 7f3bce7fc700 1 -- 172.21.15.82:0/365386959 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3bd0004020 con 0x7f3bd8101bd0 2023-07-14T19:28:14.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.555+0000 7f3bce7fc700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bc40a11d0 0x7f3bc40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.555+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bc40a11d0 0x7f3bc40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.555+0000 7f3bce7fc700 1 -- 172.21.15.82:0/365386959 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f3bd00cc220 con 0x7f3bd8101bd0 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.555+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bc40a11d0 0x7f3bc40a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009bd0 tx=0x7f3bd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 Processor -- start 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 -- start start 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d5127e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5127f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d5127fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5117ec60 con 0x563d5127ec70 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d511d19f0 con 0x563d5127e320 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d511cae80 con 0x563d5127f870 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5127f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5127f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41908/0 (socket says 172.21.15.82:41908) 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/3952575388 learned_addr learned my addr 172.21.15.82:0/3952575388 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d5127e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bddd30700 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d5127fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/3952575388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 msgr2=0x563d5127fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d5127fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/3952575388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 msgr2=0x563d5127e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d5127e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/3952575388 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d50ff8d90 con 0x563d5127ec70 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.556+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d5127e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:14.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5127f290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8015000 tx=0x7f3bc8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0229cfd00598995 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3bc37fe700 1 -- 172.21.15.82:0/3952575388 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800bee0 con 0x563d5127ec70 2023-07-14T19:28:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3bc37fe700 1 -- 172.21.15.82:0/3952575388 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bc801e040 con 0x563d5127ec70 2023-07-14T19:28:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3bc37fe700 1 -- 172.21.15.82:0/3952575388 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8004d60 con 0x563d5127ec70 2023-07-14T19:28:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 -- 172.21.15.82:0/3952575388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 msgr2=0x563d5127f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5127f290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8015000 tx=0x7f3bc8005c00 comp rx=0 tx=0).stop 2023-07-14T19:28:14.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 -- 172.21.15.82:0/3952575388 shutdown_connections 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d5127fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d5127e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/3952575388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5127f290 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 -- 172.21.15.82:0/3952575388 >> 172.21.15.82:0/3952575388 conn(0x563d5126f630 msgr2=0x563d51274980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 -- 172.21.15.82:0/3952575388 shutdown_connections 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.557+0000 7f3beb92bb80 1 -- 172.21.15.82:0/3952575388 wait complete. 2023-07-14T19:28:14.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 -- start start 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 Processor -- start 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51287cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127ec70 0x563d51288210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5116aa70 con 0x563d5127f870 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5116a770 con 0x563d5127e320 2023-07-14T19:28:14.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5104e8d0 con 0x563d5127ec70 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3beb92bb80 10 client.0 _ll_register_callbacks cb 0x563d51168710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3bddd30700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3bddd30700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41916/0 (socket says 172.21.15.82:41916) 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3bddd30700 1 -- 172.21.15.82:0/1084599270 learned_addr learned my addr 172.21.15.82:0/1084599270 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.558+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51287cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bddd30700 1 -- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127ec70 msgr2=0x563d51288210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bddd30700 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127ec70 0x563d51288210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bddd30700 1 -- 172.21.15.82:0/1084599270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 msgr2=0x563d51287cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bddd30700 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51287cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bddd30700 1 -- 172.21.15.82:0/1084599270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd8108960 con 0x563d5127f870 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bddd30700 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128f250 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0096160 tx=0x7f3bd0099050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613605f0bea974fa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/1084599270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00a1b40 con 0x563d5127f870 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/1084599270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f3bd0099950 con 0x563d5127f870 2023-07-14T19:28:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d511244a0 con 0x563d5127f870 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/1084599270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd0099b10 con 0x563d5127f870 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 10 client.15201 fetch_fsmap learned FSMap version 22 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563d50fe1c10 con 0x563d5127f870 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/1084599270 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bd00c7070 con 0x563d5127f870 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/1084599270 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3bd0004c60 con 0x563d5127f870 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 10 client.15201 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:28:14.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 20 client.15201 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d51293000 0x563d51292400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 4 client.15201 mds_command: new command op to 24481 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.559+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d51248d50 con 0x563d51293000 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.560+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d51293000 0x563d51292400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.560+0000 7f3bc0ff9700 10 client.15201 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.560+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d51293000 0x563d51292400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66c1fd7778c91cb1 server_cookie=52a651c97fef1c1f in_seq=0 out_seq=0 2023-07-14T19:28:14.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/1084599270 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563d51248d50 con 0x563d51293000 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3bc0ff9700 10 client.15201 handle_command_reply: tid=0 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 client.15201 shutdown 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 msgr2=0x563d5128f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128f250 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0096160 tx=0x7f3bd0099050 comp rx=0 tx=0).stop 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 shutdown_connections 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128f250 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51287cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d51293000 0x563d51292400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/1084599270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127ec70 0x563d51288210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3bc0ff9700 0 client.15201 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 >> 172.21.15.82:0/1084599270 conn(0x563d5126f630 msgr2=0x563d51273600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 shutdown_connections 2023-07-14T19:28:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 1 -- 172.21.15.82:0/1084599270 wait complete. 2023-07-14T19:28:14.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.561+0000 7f3beb92bb80 20 client.15201 trim_cache size 0 max 16384 2023-07-14T19:28:14.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.578+0000 7f3beb92bb80 1 Processor -- start 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.578+0000 7f3beb92bb80 1 -- start start 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51285bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5128f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d51292400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d51280160 con 0x563d5127ec70 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d51199790 con 0x563d5127e320 2023-07-14T19:28:14.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5105ade0 con 0x563d5127f870 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5128f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5128f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41920/0 (socket says 172.21.15.82:41920) 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/226006590 learned_addr learned my addr 172.21.15.82:0/226006590 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bddd30700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d51292400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51285bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/226006590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 msgr2=0x563d51292400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.582 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d51292400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/226006590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 msgr2=0x563d51285bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51285bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.583 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 -- 172.21.15.82:0/226006590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d5101f200 con 0x563d5127ec70 2023-07-14T19:28:14.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bddd30700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d51292400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51285bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5128f210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000c00 tx=0x7f3bc8009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce779be1e35b199a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bc3fff700 1 -- 172.21.15.82:0/226006590 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800bee0 con 0x563d5127ec70 2023-07-14T19:28:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bc3fff700 1 -- 172.21.15.82:0/226006590 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bc8005700 con 0x563d5127ec70 2023-07-14T19:28:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.579+0000 7f3bc3fff700 1 -- 172.21.15.82:0/226006590 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc802b7b0 con 0x563d5127ec70 2023-07-14T19:28:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 -- 172.21.15.82:0/226006590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 msgr2=0x563d5128f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5128f210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000c00 tx=0x7f3bc8009570 comp rx=0 tx=0).stop 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 -- 172.21.15.82:0/226006590 shutdown_connections 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127f870 0x563d51292400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127e320 0x563d51285bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/226006590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127ec70 0x563d5128f210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 -- 172.21.15.82:0/226006590 >> 172.21.15.82:0/226006590 conn(0x563d5126f630 msgr2=0x563d512740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 -- 172.21.15.82:0/226006590 shutdown_connections 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 -- 172.21.15.82:0/226006590 wait complete. 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 -- start start 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.580+0000 7f3beb92bb80 1 Processor -- start 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127e320 0x563d5128a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127ec70 0x563d5128af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5128e2e0 con 0x563d5127f870 2023-07-14T19:28:14.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5128e450 con 0x563d5127ec70 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d5128e5c0 con 0x563d5127e320 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3beb92bb80 10 client.0 _ll_register_callbacks cb 0x7f3bd0095a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41928/0 (socket says 172.21.15.82:41928) 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 -- 172.21.15.82:0/928317652 learned_addr learned my addr 172.21.15.82:0/928317652 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127e320 0x563d5128a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bdcd2e700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127ec70 0x563d5128af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 -- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127e320 msgr2=0x563d5128a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127e320 0x563d5128a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 -- 172.21.15.82:0/928317652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127ec70 msgr2=0x563d5128af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127ec70 0x563d5128af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 -- 172.21.15.82:0/928317652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d50ff8d90 con 0x563d5127f870 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bddd30700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128dda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0004b80 tx=0x7f3bd00a1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502cb0548b278ec3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.581+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/928317652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0004de0 con 0x563d5127f870 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/928317652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f3bd00a1b40 con 0x563d5127f870 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d511c3e60 con 0x563d5127f870 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/928317652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bd00a18a0 con 0x563d5127f870 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 10 client.15207 fetch_fsmap learned FSMap version 22 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563d50ff7260 con 0x563d5127f870 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/928317652 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d511c3e60 con 0x563d5127f870 2023-07-14T19:28:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/928317652 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3bd0098d00 con 0x563d5127f870 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 10 client.15207 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 20 client.15207 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d512ec210 0x563d512e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 4 client.15207 mds_command: new command op to 24481 tid=0[{"prefix": "status"}] 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d51186000 con 0x563d512ec210 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.582+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d512ec210 0x563d512e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3bc0ff9700 10 client.15207 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3bdd52f700 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d512ec210 0x563d512e8cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8646fe6b24b8a135 server_cookie=94dbe6388e61da11 in_seq=0 out_seq=0 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3bc0ff9700 1 -- 172.21.15.82:0/928317652 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563d51186000 con 0x563d512ec210 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3bc0ff9700 10 client.15207 handle_command_reply: tid=0 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 client.15207 shutdown 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 msgr2=0x563d5128dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128dda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0004b80 tx=0x7f3bd00a1a40 comp rx=0 tx=0).stop 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 shutdown_connections 2023-07-14T19:28:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d5127f870 0x563d5128dda0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x563d512ec210 0x563d512e8cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d5127e320 0x563d5128a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 --2- 172.21.15.82:0/928317652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d5127ec70 0x563d5128af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3bc0ff9700 0 client.15207 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 >> 172.21.15.82:0/928317652 conn(0x563d5126f630 msgr2=0x563d51285bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 shutdown_connections 2023-07-14T19:28:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 1 -- 172.21.15.82:0/928317652 wait complete. 2023-07-14T19:28:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.583+0000 7f3beb92bb80 20 client.15207 trim_cache size 0 max 16384 2023-07-14T19:28:14.601 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:14.601 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:28:14.601 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 2, 2023-07-14T19:28:14.601 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24481, 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 72.813145810999998, 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 22, 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-07-14T19:28:14.602 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-07-14T19:28:14.603 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 90.108272256999996 2023-07-14T19:28:14.603 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bc40a11d0 msgr2=0x7f3bc40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bc40a11d0 0x7f3bc40a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009bd0 tx=0x7f3bd4009260 comp rx=0 tx=0).stop 2023-07-14T19:28:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 msgr2=0x7f3bd819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd819cf20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0007ee0 tx=0x7f3bd000d3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 shutdown_connections 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f3bc40a11d0 0x7f3bc40a3680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd8101bd0 0x7f3bd819cf20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bd80fe310 0x7f3bd8195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 --2- 172.21.15.82:0/365386959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bd8101280 0x7f3bd8195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 >> 172.21.15.82:0/365386959 conn(0x7f3bd80f98a0 msgr2=0x7f3bd80fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 shutdown_connections 2023-07-14T19:28:14.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.584+0000 7f3bdf793700 1 -- 172.21.15.82:0/365386959 wait complete. 2023-07-14T19:28:14.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-14T19:28:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 -- 172.21.15.82:0/1899986544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 msgr2=0x7ffa9c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 --2- 172.21.15.82:0/1899986544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c107010 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400b0a0 tx=0x7ffa9400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.728 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 -- 172.21.15.82:0/1899986544 shutdown_connections 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 --2- 172.21.15.82:0/1899986544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c107010 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 --2- 172.21.15.82:0/1899986544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 0x7ffa9c0fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 --2- 172.21.15.82:0/1899986544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa9c0ffdc0 0x7ffa9c100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 -- 172.21.15.82:0/1899986544 >> 172.21.15.82:0/1899986544 conn(0x7ffa9c0fa1f0 msgr2=0x7ffa9c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 -- 172.21.15.82:0/1899986544 shutdown_connections 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.726+0000 7ffaa1683700 1 -- 172.21.15.82:0/1899986544 wait complete. 2023-07-14T19:28:14.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 Processor -- start 2023-07-14T19:28:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 -- start start 2023-07-14T19:28:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 0x7ffa9c194930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa9c0ffdc0 0x7ffa9c194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c19c4b0 con 0x7ffa9c100750 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c19c5f0 con 0x7ffa9c0ffdc0 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffaa1683700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c19c780 con 0x7ffa9c0febd0 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.727+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c19bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41942/0 (socket says 172.21.15.82:41942) 2023-07-14T19:28:14.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3175122681 learned_addr learned my addr 172.21.15.82:0/3175122681 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9affd700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 0x7ffa9c194930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9a7fc700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa9c0ffdc0 0x7ffa9c194e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3175122681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 msgr2=0x7ffa9c194930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 0x7ffa9c194930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3175122681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa9c0ffdc0 msgr2=0x7ffa9c194e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa9c0ffdc0 0x7ffa9c194e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3175122681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94009d00 con 0x7ffa9c100750 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9affd700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 0x7ffa9c194930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c19bee0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400b780 tx=0x7ffa9400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa83fff700 1 -- 172.21.15.82:0/3175122681 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400e040 con 0x7ffa9c100750 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c19ca00 con 0x7ffa9c100750 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa83fff700 1 -- 172.21.15.82:0/3175122681 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa94004e60 con 0x7ffa9c100750 2023-07-14T19:28:14.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffa83fff700 1 -- 172.21.15.82:0/3175122681 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9401b870 con 0x7ffa9c100750 2023-07-14T19:28:14.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.728+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c19ce80 con 0x7ffa9c100750 2023-07-14T19:28:14.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.730+0000 7ffa83fff700 1 -- 172.21.15.82:0/3175122681 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffa9401b9d0 con 0x7ffa9c100750 2023-07-14T19:28:14.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.731+0000 7ffa83fff700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ffa840a11d0 0x7ffa840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.731+0000 7ffa9affd700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ffa840a11d0 0x7ffa840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.731+0000 7ffa83fff700 1 -- 172.21.15.82:0/3175122681 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7ffa940ce340 con 0x7ffa9c100750 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.731+0000 7ffaad81bb80 1 Processor -- start 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9affd700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ffa840a11d0 0x7ffa840a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0ffc20 tx=0x7ffa8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 -- start start 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5bd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec4c1c60 con 0x55faec5be460 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec5149f0 con 0x55faec5bcf10 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec50de80 con 0x55faec5bd860 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5be8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41950/0 (socket says 172.21.15.82:41950) 2023-07-14T19:28:14.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/6841638 learned_addr learned my addr 172.21.15.82:0/6841638 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9a7fc700 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9affd700 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5bd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/6841638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 msgr2=0x55faec5bde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5bde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/6841638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 msgr2=0x55faec5bd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5bd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/6841638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faec33bd90 con 0x55faec5be460 2023-07-14T19:28:14.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9affd700 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5be8d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa94097d00 tx=0x7ffa9401dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98077b26c630f5c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa817fa700 1 -- 172.21.15.82:0/6841638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940a3bc0 con 0x55faec5be460 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa817fa700 1 -- 172.21.15.82:0/6841638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa940cbf70 con 0x55faec5be460 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.732+0000 7ffa817fa700 1 -- 172.21.15.82:0/6841638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940bb2a0 con 0x55faec5be460 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 -- 172.21.15.82:0/6841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 msgr2=0x55faec5be8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5be8d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa94097d00 tx=0x7ffa9401dea0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 -- 172.21.15.82:0/6841638 shutdown_connections 2023-07-14T19:28:14.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5be8d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5bd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/6841638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 -- 172.21.15.82:0/6841638 >> 172.21.15.82:0/6841638 conn(0x55faec5b2630 msgr2=0x55faec5b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 -- 172.21.15.82:0/6841638 shutdown_connections 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 -- 172.21.15.82:0/6841638 wait complete. 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.733+0000 7ffaad81bb80 1 -- start start 2023-07-14T19:28:14.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 Processor -- start 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec4ada70 con 0x55faec5be460 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec4ad770 con 0x55faec5bcf10 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec3918d0 con 0x55faec5bd860 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffaad81bb80 10 client.0 _ll_register_callbacks cb 0x55faec4ab710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:14.745 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41956/0 (socket says 172.21.15.82:41956) 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/1397281781 learned_addr learned my addr 172.21.15.82:0/1397281781 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9a7fc700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9affd700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/1397281781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 msgr2=0x55faec5c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 msgr2=0x55faec5c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.746 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/1397281781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c19ce80 con 0x55faec5be460 2023-07-14T19:28:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9affd700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5cdc50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009fd0 tx=0x7ffa9400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fa42a72a75c2c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.734+0000 7ffa7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffa7eff5700 1 -- 172.21.15.82:0/1397281781 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940bb2a0 con 0x55faec5be460 2023-07-14T19:28:14.747 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffa7eff5700 1 -- 172.21.15.82:0/1397281781 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7ffa940a3bc0 con 0x55faec5be460 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faec4674a0 con 0x55faec5be460 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffa7eff5700 1 -- 172.21.15.82:0/1397281781 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa9409bc70 con 0x55faec5be460 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 10 client.15219 fetch_fsmap learned FSMap version 22 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55faec324c10 con 0x55faec5be460 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffa7eff5700 1 -- 172.21.15.82:0/1397281781 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55faec4674a0 con 0x55faec5be460 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffa7eff5700 1 -- 172.21.15.82:0/1397281781 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffa9409fa90 con 0x55faec5be460 2023-07-14T19:28:14.748 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 10 client.15219 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 20 client.15219 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec5d1a60 0x55faec5d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 4 client.15219 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.735+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faec58bd50 con 0x55faec5d1a60 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.736+0000 7ffa9affd700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec5d1a60 0x55faec5d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.736+0000 7ffa7eff5700 10 client.15219 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:14.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.736+0000 7ffa9affd700 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec5d1a60 0x55faec5d0af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b93faeb8a2d3f7b0 server_cookie=6192e7dd611ec936 in_seq=0 out_seq=0 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffa7eff5700 1 -- 172.21.15.82:0/1397281781 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55faec58bd50 con 0x55faec5d1a60 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffa7eff5700 10 client.15219 handle_command_reply: tid=0 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 client.15219 shutdown 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 msgr2=0x55faec5cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5cdc50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009fd0 tx=0x7ffa9400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 shutdown_connections 2023-07-14T19:28:14.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec5d1a60 0x55faec5d0af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5cdc50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/1397281781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 >> 172.21.15.82:0/1397281781 conn(0x55faec5b2630 msgr2=0x55faec5b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffa7eff5700 0 client.15219 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 shutdown_connections 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 1 -- 172.21.15.82:0/1397281781 wait complete. 2023-07-14T19:28:14.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.737+0000 7ffaad81bb80 20 client.15219 trim_cache size 0 max 16384 2023-07-14T19:28:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 Processor -- start 2023-07-14T19:28:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 -- start start 2023-07-14T19:28:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec58bd50 con 0x55faec5be460 2023-07-14T19:28:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec4674a0 con 0x55faec5bcf10 2023-07-14T19:28:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec5b8c80 con 0x55faec5bd860 2023-07-14T19:28:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41964/0 (socket says 172.21.15.82:41964) 2023-07-14T19:28:14.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3905293973 learned_addr learned my addr 172.21.15.82:0/3905293973 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3905293973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 msgr2=0x55faec5c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3905293973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 msgr2=0x55faec5c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 -- 172.21.15.82:0/3905293973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faec362200 con 0x55faec5be460 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.755+0000 7ffa9b7fe700 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5c6480 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffa9409b2d0 tx=0x7ffa94009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2279e57657a8e02 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffa81ffb700 1 -- 172.21.15.82:0/3905293973 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9409bc70 con 0x55faec5be460 2023-07-14T19:28:14.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffa81ffb700 1 -- 172.21.15.82:0/3905293973 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa940a3bc0 con 0x55faec5be460 2023-07-14T19:28:14.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffa81ffb700 1 -- 172.21.15.82:0/3905293973 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940a3950 con 0x55faec5be460 2023-07-14T19:28:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 -- 172.21.15.82:0/3905293973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 msgr2=0x55faec5c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5c6480 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffa9409b2d0 tx=0x7ffa94009040 comp rx=0 tx=0).stop 2023-07-14T19:28:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 -- 172.21.15.82:0/3905293973 shutdown_connections 2023-07-14T19:28:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5be460 0x55faec5c6480 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/3905293973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5bd860 0x55faec5c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 -- 172.21.15.82:0/3905293973 >> 172.21.15.82:0/3905293973 conn(0x55faec5b2630 msgr2=0x55faec5c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 -- 172.21.15.82:0/3905293973 shutdown_connections 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 -- 172.21.15.82:0/3905293973 wait complete. 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 -- start start 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.756+0000 7ffaad81bb80 1 Processor -- start 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 0x55faec6250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5be460 0x55faec627600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec5ce0f0 con 0x55faec5bd860 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec5b7bc0 con 0x55faec5bcf10 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec5b7d80 con 0x55faec5be460 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 10 client.0 _ll_register_callbacks cb 0x55faec4ab710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 0x55faec6250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 0x55faec6250b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41968/0 (socket says 172.21.15.82:41968) 2023-07-14T19:28:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 -- 172.21.15.82:0/860940295 learned_addr learned my addr 172.21.15.82:0/860940295 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9affd700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 -- 172.21.15.82:0/860940295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5be460 msgr2=0x55faec627600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5be460 0x55faec627600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 -- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 msgr2=0x55faec5b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 -- 172.21.15.82:0/860940295 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55faec324c10 con 0x55faec5bd860 2023-07-14T19:28:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9affd700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa9a7fc700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 0x55faec6250b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa900136e0 tx=0x7ffa90013aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a13cfc62d13ebd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa7eff5700 1 -- 172.21.15.82:0/860940295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90018070 con 0x55faec5bd860 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffa7eff5700 1 -- 172.21.15.82:0/860940295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7ffa90011650 con 0x55faec5bd860 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.757+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa940cbf70 con 0x55faec5bd860 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffa7eff5700 1 -- 172.21.15.82:0/860940295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa90011810 con 0x55faec5bd860 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 10 client.15225 fetch_fsmap learned FSMap version 22 2023-07-14T19:28:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55faec33a260 con 0x55faec5bd860 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffa7eff5700 1 -- 172.21.15.82:0/860940295 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa940cbf70 con 0x55faec5bd860 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffa7eff5700 1 -- 172.21.15.82:0/860940295 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ffa9000fae0 con 0x55faec5bd860 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 10 client.15225 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 10 client.15225 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 20 client.15225 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec62c660 0x55faec628ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 4 client.15225 mds_command: new command op to 24491 tid=0[{"prefix": "status"}] 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55faec506e60 con 0x55faec62c660 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.758+0000 7ffa9affd700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec62c660 0x55faec628ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffa9affd700 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec62c660 0x55faec628ff0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6576b074109c0dda server_cookie=841df549852fd6cf in_seq=0 out_seq=0 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffa7eff5700 10 client.15225 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffa7eff5700 1 -- 172.21.15.82:0/860940295 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55faec506e60 con 0x55faec62c660 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffa7eff5700 10 client.15225 handle_command_reply: tid=0 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 client.15225 shutdown 2023-07-14T19:28:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 msgr2=0x55faec6250b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 0x55faec6250b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffa900136e0 tx=0x7ffa90013aa0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 shutdown_connections 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x55faec62c660 0x55faec628ff0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55faec5be460 0x55faec627600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55faec5bcf10 0x55faec5b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffa7eff5700 0 client.15225 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 --2- 172.21.15.82:0/860940295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55faec5bd860 0x55faec6250b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 >> 172.21.15.82:0/860940295 conn(0x55faec5b2630 msgr2=0x55faec5c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 shutdown_connections 2023-07-14T19:28:14.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 1 -- 172.21.15.82:0/860940295 wait complete. 2023-07-14T19:28:14.771 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.759+0000 7ffaad81bb80 20 client.15225 trim_cache size 0 max 16384 2023-07-14T19:28:14.771 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:28:14.771 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 1, 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24491, 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 73.992308856999998, 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 22, 2023-07-14T19:28:14.772 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-07-14T19:28:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-07-14T19:28:14.773 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 92.439113104 2023-07-14T19:28:14.773 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ffa840a11d0 msgr2=0x7ffa840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ffa840a11d0 0x7ffa840a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0ffc20 tx=0x7ffa8c005f90 comp rx=0 tx=0).stop 2023-07-14T19:28:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 msgr2=0x7ffa9c19bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c19bee0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400b780 tx=0x7ffa9400bdb0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 shutdown_connections 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ffa840a11d0 0x7ffa840a3680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa9c100750 0x7ffa9c19bee0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa9c0febd0 0x7ffa9c194930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 --2- 172.21.15.82:0/3175122681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa9c0ffdc0 0x7ffa9c194e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 >> 172.21.15.82:0/3175122681 conn(0x7ffa9c0fa1f0 msgr2=0x7ffa9c103250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 shutdown_connections 2023-07-14T19:28:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.760+0000 7ffaa1683700 1 -- 172.21.15.82:0/3175122681 wait complete. 2023-07-14T19:28:14.776 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-14T19:28:14.899 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 -- 172.21.15.82:0/780698346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 msgr2=0x7fae80100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 --2- 172.21.15.82:0/780698346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80100150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae68009990 tx=0x7fae68009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 -- 172.21.15.82:0/780698346 shutdown_connections 2023-07-14T19:28:14.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 --2- 172.21.15.82:0/780698346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae80100690 0x7fae80106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.900 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 --2- 172.21.15.82:0/780698346 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae800feb10 0x7fae800fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 --2- 172.21.15.82:0/780698346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80100150 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 -- 172.21.15.82:0/780698346 >> 172.21.15.82:0/780698346 conn(0x7fae800fa130 msgr2=0x7fae800fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 -- 172.21.15.82:0/780698346 shutdown_connections 2023-07-14T19:28:14.901 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.898+0000 7fae84b43700 1 -- 172.21.15.82:0/780698346 wait complete. 2023-07-14T19:28:14.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 Processor -- start 2023-07-14T19:28:14.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 -- start start 2023-07-14T19:28:14.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae800feb10 0x7fae801948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.903 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae80100690 0x7fae8019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8019c460 con 0x7fae800ffd00 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8019c5a0 con 0x7fae800feb10 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae84b43700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8019c730 con 0x7fae80100690 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae7dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae7dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80194e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41982/0 (socket says 172.21.15.82:41982) 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae7dd9b700 1 -- 172.21.15.82:0/3673069587 learned_addr learned my addr 172.21.15.82:0/3673069587 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.904 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.899+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae800feb10 0x7fae801948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7dd9b700 1 -- 172.21.15.82:0/3673069587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae80100690 msgr2=0x7fae8019be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae80100690 0x7fae8019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae80100690 0x7fae8019be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7dd9b700 1 -- 172.21.15.82:0/3673069587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae800feb10 msgr2=0x7fae801948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae800feb10 0x7fae801948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7dd9b700 1 -- 172.21.15.82:0/3673069587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680095f0 con 0x7fae800ffd00 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae800feb10 0x7fae801948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:14.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80194e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae68004760 tx=0x7fae68009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae777fe700 1 -- 172.21.15.82:0/3673069587 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6801d070 con 0x7fae800ffd00 2023-07-14T19:28:14.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8019c9b0 con 0x7fae800ffd00 2023-07-14T19:28:14.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae777fe700 1 -- 172.21.15.82:0/3673069587 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae68004200 con 0x7fae800ffd00 2023-07-14T19:28:14.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae777fe700 1 -- 172.21.15.82:0/3673069587 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680049f0 con 0x7fae800ffd00 2023-07-14T19:28:14.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.900+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8019ce60 con 0x7fae800ffd00 2023-07-14T19:28:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.902+0000 7fae777fe700 1 -- 172.21.15.82:0/3673069587 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fae68003d40 con 0x7fae800ffd00 2023-07-14T19:28:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.903+0000 7fae777fe700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fae6c0a1150 0x7fae6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.903+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fae6c0a1150 0x7fae6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.903+0000 7fae777fe700 1 -- 172.21.15.82:0/3673069587 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fae680cd840 con 0x7fae800ffd00 2023-07-14T19:28:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.903+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fae6c0a1150 0x7fae6c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae800ffb60 tx=0x7fae70009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 Processor -- start 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 -- start start 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8df10 0x5598faf8e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 0x5598faf8ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598faf8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fae92c60 con 0x5598faf8e860 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faee59f0 con 0x5598faf8df10 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faedee80 con 0x5598faf8f460 2023-07-14T19:28:14.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae7dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 0x5598faf8ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae7dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 0x5598faf8ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41990/0 (socket says 172.21.15.82:41990) 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae7dd9b700 1 -- 172.21.15.82:0/1295994727 learned_addr learned my addr 172.21.15.82:0/1295994727 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.904+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598faf8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8df10 0x5598faf8e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7dd9b700 1 -- 172.21.15.82:0/1295994727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 msgr2=0x5598faf8f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598faf8f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7dd9b700 1 -- 172.21.15.82:0/1295994727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8df10 msgr2=0x5598faf8e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8df10 0x5598faf8e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7dd9b700 1 -- 172.21.15.82:0/1295994727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598fad0cd90 con 0x5598faf8e860 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 0x5598faf8ee80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae68097770 tx=0x7fae6800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b02c2ca987b4193 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae74ff9700 1 -- 172.21.15.82:0/1295994727 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680a30c0 con 0x5598faf8e860 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae74ff9700 1 -- 172.21.15.82:0/1295994727 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae680990a0 con 0x5598faf8e860 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.905+0000 7fae74ff9700 1 -- 172.21.15.82:0/1295994727 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68099c60 con 0x5598faf8e860 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1295994727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 msgr2=0x5598faf8ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 0x5598faf8ee80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae68097770 tx=0x7fae6800b540 comp rx=0 tx=0).stop 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1295994727 shutdown_connections 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598faf8f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8df10 0x5598faf8e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1295994727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8e860 0x5598faf8ee80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1295994727 >> 172.21.15.82:0/1295994727 conn(0x5598faf83630 msgr2=0x5598faf88980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1295994727 shutdown_connections 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1295994727 wait complete. 2023-07-14T19:28:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- start start 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 Processor -- start 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8df10 0x5598faf97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598faf97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 0x5598faf9eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fae7ea70 con 0x5598faf8f460 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fae7e770 con 0x5598faf8e860 2023-07-14T19:28:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598fad628d0 con 0x5598faf8df10 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae90cdbb80 10 client.0 _ll_register_callbacks cb 0x5598fae7c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae7ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 0x5598faf9eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae7ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 0x5598faf9eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41998/0 (socket says 172.21.15.82:41998) 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae7ed9d700 1 -- 172.21.15.82:0/1459053269 learned_addr learned my addr 172.21.15.82:0/1459053269 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.906+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8df10 0x5598faf97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598faf97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7ed9d700 1 -- 172.21.15.82:0/1459053269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8df10 msgr2=0x5598faf97590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8df10 0x5598faf97590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7ed9d700 1 -- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 msgr2=0x5598faf97ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598faf97ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7ed9d700 1 -- 172.21.15.82:0/1459053269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8019ce60 con 0x5598faf8f460 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 0x5598faf9eb10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae78010670 tx=0x7fae78010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4847f09834eaa4ad server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae65ffb700 1 -- 172.21.15.82:0/1459053269 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78012650 con 0x5598faf8f460 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598fae384a0 con 0x5598faf8f460 2023-07-14T19:28:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae65ffb700 1 -- 172.21.15.82:0/1459053269 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fae780127b0 con 0x5598faf8f460 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae65ffb700 1 -- 172.21.15.82:0/1459053269 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae78012970 con 0x5598faf8f460 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae90cdbb80 10 client.15237 fetch_fsmap learned FSMap version 22 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5598facf5c10 con 0x5598faf8f460 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.907+0000 7fae65ffb700 1 -- 172.21.15.82:0/1459053269 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5598fae384a0 con 0x5598faf8f460 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae65ffb700 1 -- 172.21.15.82:0/1459053269 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae78027c70 con 0x5598faf8f460 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae90cdbb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:28:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae90cdbb80 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae90cdbb80 20 client.15237 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598fafa2910 0x5598fafa1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae90cdbb80 4 client.15237 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598faf5cd50 con 0x5598fafa2910 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598fafa2910 0x5598fafa1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae65ffb700 10 client.15237 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.908+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598fafa2910 0x5598fafa1cc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff1a4d883383857 server_cookie=131d471b01d9b410 in_seq=0 out_seq=0 2023-07-14T19:28:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae65ffb700 1 -- 172.21.15.82:0/1459053269 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5598faf5cd50 con 0x5598fafa2910 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae65ffb700 10 client.15237 handle_command_reply: tid=0 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 client.15237 shutdown 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 msgr2=0x5598faf9eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 0x5598faf9eb10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae78010670 tx=0x7fae78010980 comp rx=0 tx=0).stop 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 shutdown_connections 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8f460 0x5598faf9eb10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598fafa2910 0x5598fafa1cc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8df10 0x5598faf97590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae65ffb700 0 client.15237 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1459053269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598faf97ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 >> 172.21.15.82:0/1459053269 conn(0x5598faf83630 msgr2=0x5598faf87600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 shutdown_connections 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1459053269 wait complete. 2023-07-14T19:28:14.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.909+0000 7fae90cdbb80 20 client.15237 trim_cache size 0 max 16384 2023-07-14T19:28:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 Processor -- start 2023-07-14T19:28:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 -- start start 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598fafa1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598fafa2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598fafab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faf97160 con 0x5598faf8df10 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faee59f0 con 0x5598faf8e860 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faf5cd50 con 0x5598faf8f460 2023-07-14T19:28:14.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598fafa1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598fafa1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42002/0 (socket says 172.21.15.82:42002) 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 -- 172.21.15.82:0/1812448228 learned_addr learned my addr 172.21.15.82:0/1812448228 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598fafab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598fafa2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 -- 172.21.15.82:0/1812448228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 msgr2=0x5598fafab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598fafab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 -- 172.21.15.82:0/1812448228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 msgr2=0x5598fafa2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598fafa2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7e59c700 1 -- 172.21.15.82:0/1812448228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598fad33200 con 0x5598faf8df10 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.927+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598fafa2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae7e59c700 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598fafa1cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fae70009bb0 tx=0x7fae70015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eb8eee963dc0bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae757fa700 1 -- 172.21.15.82:0/1812448228 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70016df0 con 0x5598faf8df10 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae757fa700 1 -- 172.21.15.82:0/1812448228 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae70004020 con 0x5598faf8df10 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae757fa700 1 -- 172.21.15.82:0/1812448228 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70031680 con 0x5598faf8df10 2023-07-14T19:28:14.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1812448228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 msgr2=0x5598fafa1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598fafa1cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fae70009bb0 tx=0x7fae70015000 comp rx=0 tx=0).stop 2023-07-14T19:28:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1812448228 shutdown_connections 2023-07-14T19:28:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8f460 0x5598fafab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598fafa1cc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/1812448228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8e860 0x5598fafa2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1812448228 >> 172.21.15.82:0/1812448228 conn(0x5598faf83630 msgr2=0x5598faf95790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1812448228 shutdown_connections 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.928+0000 7fae90cdbb80 1 -- 172.21.15.82:0/1812448228 wait complete. 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 -- start start 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 Processor -- start 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598faf88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8e860 0x5598fafab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8f460 0x5598faff6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faff6c60 con 0x5598faf8df10 2023-07-14T19:28:14.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faff6dd0 con 0x5598faf8f460 2023-07-14T19:28:14.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598faff6f90 con 0x5598faf8e860 2023-07-14T19:28:14.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae90cdbb80 10 client.0 _ll_register_callbacks cb 0x7fae7800ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:14.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598faf88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598faf88a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42008/0 (socket says 172.21.15.82:42008) 2023-07-14T19:28:14.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 -- 172.21.15.82:0/3273090911 learned_addr learned my addr 172.21.15.82:0/3273090911 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8e860 0x5598fafab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 -- 172.21.15.82:0/3273090911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8e860 msgr2=0x5598fafab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8e860 0x5598fafab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 -- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8f460 msgr2=0x5598faff6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7ed9d700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8f460 0x5598faff6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8f460 0x5598faff6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 -- 172.21.15.82:0/3273090911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7001e040 con 0x5598faf8df10 2023-07-14T19:28:14.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae7e59c700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598faf88a70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fae70007fd0 tx=0x7fae70004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595113324e43c650 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.929+0000 7fae65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae65ffb700 1 -- 172.21.15.82:0/3273090911 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7002a030 con 0x5598faf8df10 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae65ffb700 1 -- 172.21.15.82:0/3273090911 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fae7002b070 con 0x5598faf8df10 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae680990a0 con 0x5598faf8df10 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae65ffb700 1 -- 172.21.15.82:0/3273090911 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae70004500 con 0x5598faf8df10 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 10 client.15243 fetch_fsmap learned FSMap version 22 2023-07-14T19:28:14.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5598fad0b260 con 0x5598faf8df10 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae65ffb700 1 -- 172.21.15.82:0/3273090911 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae680990a0 con 0x5598faf8df10 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae65ffb700 1 -- 172.21.15.82:0/3273090911 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae70031420 con 0x5598faf8df10 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 10 client.15243 fetch_fsmap finished waiting for FSMap version 22 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 20 client.15243 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598faffbc10 0x5598faff8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 4 client.15243 mds_command: new command op to 24524 tid=0[{"prefix": "status"}] 2023-07-14T19:28:14.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598faed7e60 con 0x5598faffbc10 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.930+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598faffbc10 0x5598faff8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae65ffb700 10 client.15243 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae7dd9b700 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598faffbc10 0x5598faff8600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7341f3b490c78e25 server_cookie=12b1a34ecdeaf6aa in_seq=0 out_seq=0 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae65ffb700 1 -- 172.21.15.82:0/3273090911 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5598faed7e60 con 0x5598faffbc10 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae65ffb700 10 client.15243 handle_command_reply: tid=0 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 client.15243 shutdown 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 msgr2=0x5598faf88a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598faf88a70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fae70007fd0 tx=0x7fae70004260 comp rx=0 tx=0).stop 2023-07-14T19:28:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 shutdown_connections 2023-07-14T19:28:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598faf8f460 0x5598faff6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5598faf8df10 0x5598faf88a70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5598faffbc10 0x5598faff8600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 --2- 172.21.15.82:0/3273090911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5598faf8e860 0x5598fafab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae65ffb700 0 client.15243 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 >> 172.21.15.82:0/3273090911 conn(0x5598faf83630 msgr2=0x5598fafa1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 shutdown_connections 2023-07-14T19:28:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.931+0000 7fae90cdbb80 1 -- 172.21.15.82:0/3273090911 wait complete. 2023-07-14T19:28:14.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae90cdbb80 20 client.15243 trim_cache size 0 max 16384 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "4f3eb35e-227b-11ee-9b33-001a4aab830c", 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 24524, 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2023-07-14T19:28:14.945 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2023-07-14T19:28:14.946 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 77.192275498000001, 2023-07-14T19:28:14.946 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 22, 2023-07-14T19:28:14.946 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 83, 2023-07-14T19:28:14.946 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 81, 2023-07-14T19:28:14.946 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 84.033070649999999 2023-07-14T19:28:14.946 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:14.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fae6c0a1150 msgr2=0x7fae6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fae6c0a1150 0x7fae6c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae800ffb60 tx=0x7fae70009040 comp rx=0 tx=0).stop 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 msgr2=0x7fae80194e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80194e20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae68004760 tx=0x7fae68009f90 comp rx=0 tx=0).stop 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 shutdown_connections 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae80100690 0x7fae8019be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae800feb10 0x7fae801948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fae6c0a1150 0x7fae6c0a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 --2- 172.21.15.82:0/3673069587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae800ffd00 0x7fae80194e20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 >> 172.21.15.82:0/3673069587 conn(0x7fae800fa130 msgr2=0x7fae80103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 shutdown_connections 2023-07-14T19:28:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:14.932+0000 7fae84b43700 1 -- 172.21.15.82:0/3673069587 wait complete. 2023-07-14T19:28:14.949 INFO:teuthology.run_tasks:Running task kclient... 2023-07-14T19:28:14.973 INFO:tasks.kclient:Mounting kernel clients... 2023-07-14T19:28:14.974 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-14T19:28:14.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:14.975 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2023-07-14T19:28:14.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:14.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2023-07-14T19:28:15.055 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:14 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4028853883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:15.056 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:14 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3490599342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:15.061 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2023-07-14T19:28:15.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:28:15.062 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-07-14T19:28:15.063 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-14T19:28:15.063 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:15.063 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph fs ls 2023-07-14T19:28:15.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:14 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4028853883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:15.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:14 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3490599342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:14 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4028853883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:14 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3490599342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:15.648 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:28:16.151 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:15 smithi082 ceph-mon[127461]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:16.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:15 smithi133 ceph-mon[128894]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:16.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:15 smithi161 ceph-mon[129184]: pgmap v288: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-07-14T19:28:16.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 -- 172.21.15.82:0/3658047208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 msgr2=0x7f19ec0ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:16.863 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 --2- 172.21.15.82:0/3658047208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec0ff310 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19e80099c0 tx=0x7f19e8009cd0 comp rx=0 tx=0).stop 2023-07-14T19:28:16.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 -- 172.21.15.82:0/3658047208 shutdown_connections 2023-07-14T19:28:16.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 --2- 172.21.15.82:0/3658047208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101150 0x7f19ec10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:16.864 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 --2- 172.21.15.82:0/3658047208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec0ff310 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:16.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 --2- 172.21.15.82:0/3658047208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19ec1007a0 0x7f19ec100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:16.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 -- 172.21.15.82:0/3658047208 >> 172.21.15.82:0/3658047208 conn(0x7f19ec075c90 msgr2=0x7f19ec076090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:16.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 -- 172.21.15.82:0/3658047208 shutdown_connections 2023-07-14T19:28:16.865 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.861+0000 7f19f3427700 1 -- 172.21.15.82:0/3658047208 wait complete. 2023-07-14T19:28:16.868 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.862+0000 7f19f3427700 1 Processor -- start 2023-07-14T19:28:16.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.862+0000 7f19f3427700 1 -- start start 2023-07-14T19:28:16.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.862+0000 7f19f3427700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec194d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:16.869 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.862+0000 7f19f3427700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19ec1007a0 0x7f19ec195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:16.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f3427700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101150 0x7f19ec1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:16.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f3427700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec198bb0 con 0x7f19ec0fef40 2023-07-14T19:28:16.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f3427700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec198d20 con 0x7f19ec1007a0 2023-07-14T19:28:16.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f3427700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec198ee0 con 0x7f19ec101150 2023-07-14T19:28:16.870 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec194d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:16.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec194d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42018/0 (socket says 172.21.15.82:42018) 2023-07-14T19:28:16.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 -- 172.21.15.82:0/248671252 learned_addr learned my addr 172.21.15.82:0/248671252 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:16.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f09c2700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19ec1007a0 0x7f19ec195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:16.871 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 -- 172.21.15.82:0/248671252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101150 msgr2=0x7f19ec1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:16.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101150 0x7f19ec1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:16.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 -- 172.21.15.82:0/248671252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19ec1007a0 msgr2=0x7f19ec195240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:16.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19ec1007a0 0x7f19ec195240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:16.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 -- 172.21.15.82:0/248671252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009520 con 0x7f19ec0fef40 2023-07-14T19:28:16.872 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f11c3700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec194d00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b580 tx=0x7f19e801a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e801f070 con 0x7f19ec0fef40 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e80054f0 con 0x7f19ec0fef40 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8004bd0 con 0x7f19ec0fef40 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e80095f0 con 0x7f19ec0fef40 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.863+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec199330 con 0x7f19ec0fef40 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.864+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec04e340 con 0x7f19ec0fef40 2023-07-14T19:28:16.873 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.865+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f19e8023070 con 0x7f19ec0fef40 2023-07-14T19:28:16.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.866+0000 7f19e27fc700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d80a11d0 0x7f19d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:16.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.866+0000 7f19f09c2700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d80a11d0 0x7f19d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:16.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.866+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f19e80cf6e0 con 0x7f19ec0fef40 2023-07-14T19:28:16.874 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.866+0000 7f19f09c2700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d80a11d0 0x7f19d80a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19ec196290 tx=0x7f19dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:16.876 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:16.870+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f19e8098f80 con 0x7f19ec0fef40 2023-07-14T19:28:17.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.103+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f19ec068bb0 con 0x7f19ec0fef40 2023-07-14T19:28:17.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.103+0000 7f19e27fc700 1 -- 172.21.15.82:0/248671252 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f19e809c5a0 con 0x7f19ec0fef40 2023-07-14T19:28:17.105 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-14T19:28:17.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d80a11d0 msgr2=0x7f19d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:17.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d80a11d0 0x7f19d80a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19ec196290 tx=0x7f19dc009260 comp rx=0 tx=0).stop 2023-07-14T19:28:17.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 msgr2=0x7f19ec194d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:17.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec194d00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19e800b580 tx=0x7f19e801a8f0 comp rx=0 tx=0).stop 2023-07-14T19:28:17.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 shutdown_connections 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19ec101150 0x7f19ec1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f19d80a11d0 0x7f19d80a3680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ec0fef40 0x7f19ec194d00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 --2- 172.21.15.82:0/248671252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19ec1007a0 0x7f19ec195240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 >> 172.21.15.82:0/248671252 conn(0x7f19ec075c90 msgr2=0x7f19ec0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 shutdown_connections 2023-07-14T19:28:17.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:17.105+0000 7f19f3427700 1 -- 172.21.15.82:0/248671252 wait complete. 2023-07-14T19:28:17.506 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-14T19:28:17.506 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-14T19:28:17.506 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2023-07-14T19:28:17.506 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-14T19:28:17.507 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:17.507 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-14T19:28:17.507 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-14T19:28:17.507 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-14T19:28:17.507 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-14T19:28:17.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:17.508 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2023-07-14T19:28:17.526 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-14T19:28:17.527 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-14T19:28:17.527 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2023-07-14T19:28:17.527 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2023-07-14T19:28:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2023-07-14T19:28:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2023-07-14T19:28:17.528 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T19:28:17.528 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2023-07-14T19:28:17.529 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T19:28:17.529 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2023-07-14T19:28:17.529 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-14T19:28:17.530 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2023-07-14T19:28:17.530 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-14T19:28:17.530 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2023-07-14T19:28:17.530 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-14T19:28:17.531 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41287sec preferred_lft 41287sec 2023-07-14T19:28:17.531 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::1e0c:ee15:690e:98ae/64 scope link noprefixroute 2023-07-14T19:28:17.531 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2023-07-14T19:28:17.532 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-14T19:28:17.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:17.533 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-07-14T19:28:17.533 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-07-14T19:28:17.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2023-07-14T19:28:17.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2023-07-14T19:28:17.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2023-07-14T19:28:17.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-14T19:28:17.534 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-07-14T19:28:17.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:17.689 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-14T19:28:17.716 INFO:teuthology.orchestra.run.smithi082.stdout:1 2023-07-14T19:28:17.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:17.718 DEBUG:teuthology.orchestra.run.smithi082:> route 2023-07-14T19:28:17.786 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2023-07-14T19:28:17.787 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-14T19:28:17.787 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-14T19:28:17.787 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-14T19:28:17.787 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-14T19:28:17.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:17.788 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-07-14T19:28:17.788 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-07-14T19:28:17.788 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-14T19:28:17.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-14T19:28:17.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-14T19:28:17.789 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-07-14T19:28:17.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:17 smithi082 ceph-mon[127461]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T19:28:17.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:17 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/248671252' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T19:28:18.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.046 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2023-07-14T19:28:18.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.103 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2023-07-14T19:28:18.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.159 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-07-14T19:28:18.159 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-07-14T19:28:18.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T19:28:18.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-14T19:28:18.160 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-07-14T19:28:18.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:17 smithi133 ceph-mon[128894]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T19:28:18.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:17 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/248671252' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T19:28:18.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:17 smithi161 ceph-mon[129184]: pgmap v289: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-14T19:28:18.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:17 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/248671252' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-14T19:28:18.256 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-14T19:28:18.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.258 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-07-14T19:28:18.258 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-07-14T19:28:18.259 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-14T19:28:18.259 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-14T19:28:18.260 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-14T19:28:18.261 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-14T19:28:18.262 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-14T19:28:18.262 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-07-14T19:28:18.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.470 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2023-07-14T19:28:18.470 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2023-07-14T19:28:18.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2023-07-14T19:28:18.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2023-07-14T19:28:18.472 DEBUG:teuthology.orchestra.run.smithi082:> ') 2023-07-14T19:28:18.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T19:28:18.571 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:18.587 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-14T19:28:18.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-14T19:28:18.588 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:18.645 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-14T19:28:18.645 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-14T19:28:18.645 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505 2023-07-14T19:28:18.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.646 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-07-14T19:28:18.745 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-07-14T19:28:18.746 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2023-07-14T19:28:18.890 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-07-14T19:28:18.891 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2023-07-14T19:28:18.891 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.133:3300,172.21.15.161:3300" 2023-07-14T19:28:18.891 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with new device syntax: 0@4f3eb35e-227b-11ee-9b33-001a4aab830c.cephfs=/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505 2023-07-14T19:28:18.891 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.82:3300/172.21.15.133:3300/172.21.15.161:3300" will pass to kernel 2023-07-14T19:28:18.891 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: v2 mount support check returned 2 2023-07-14T19:28:18.891 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: kernel does not support v2 syntax 2023-07-14T19:28:18.892 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:3300,172.21.15.133:3300,172.21.15.161:3300:/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505 2023-07-14T19:28:18.892 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mds_namespace=cephfs,fsid=4f3eb35e-227b-11ee-9b33-001a4aab830c" will pass to kernel 2023-07-14T19:28:18.899 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-14T19:28:18.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:18.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:18.953 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T19:28:18.984 INFO:teuthology.orchestra.run.smithi082.stdout:client15252 2023-07-14T19:28:18.985 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:28:18.985 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T19:28:19.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:19.045 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T19:28:19.117 INFO:teuthology.orchestra.run.smithi082.stdout:client15252 2023-07-14T19:28:19.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:19.119 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-14T19:28:19.188 INFO:teuthology.orchestra.run.smithi082.stdout:client15252 2023-07-14T19:28:19.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:28:19.189 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c.client15252/status') 2023-07-14T19:28:19.270 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.15252 (3)192.168.0.1:0/3778204205 2023-07-14T19:28:19.270 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2023-07-14T19:28:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2023-07-14T19:28:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2023-07-14T19:28:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:70 bytes copied, 2.2723e-05 s, 3.1 MB/s 2023-07-14T19:28:19.272 INFO:teuthology.run_tasks:Running task exec... 2023-07-14T19:28:19.284 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T19:28:19.284 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:28:19.285 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2023-07-14T19:28:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- 172.21.15.82:0/3759138437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8105990 msgr2=0x7f5aa8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:19.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- 172.21.15.82:0/3759138437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8105990 0x7f5aa8107d70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009ec0 tx=0x7f5aa000b330 comp rx=0 tx=0).stop 2023-07-14T19:28:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- 172.21.15.82:0/3759138437 shutdown_connections 2023-07-14T19:28:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- 172.21.15.82:0/3759138437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8105990 0x7f5aa8107d70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- 172.21.15.82:0/3759138437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5aa8100750 0x7f5aa8102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- 172.21.15.82:0/3759138437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aa8103070 0x7f5aa8105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- 172.21.15.82:0/3759138437 >> 172.21.15.82:0/3759138437 conn(0x7f5aa80fa130 msgr2=0x7f5aa80fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:19.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- 172.21.15.82:0/3759138437 shutdown_connections 2023-07-14T19:28:19.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- 172.21.15.82:0/3759138437 wait complete. 2023-07-14T19:28:19.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 Processor -- start 2023-07-14T19:28:19.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- start start 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aa8100750 0x7f5aa80fe700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 0x7f5aa80fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5aa8105990 0x7f5aa80ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa819f980 con 0x7f5aa8103070 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa819fac0 con 0x7f5aa8100750 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.438+0000 7f5aaf540700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa819fc00 con 0x7f5aa8105990 2023-07-14T19:28:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 0x7f5aa80fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 0x7f5aa80fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42040/0 (socket says 172.21.15.82:42040) 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 -- 172.21.15.82:0/1188546454 learned_addr learned my addr 172.21.15.82:0/1188546454 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aad2dc700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aa8100750 0x7f5aa80fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aadadd700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5aa8105990 0x7f5aa80ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 -- 172.21.15.82:0/1188546454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5aa8105990 msgr2=0x7f5aa80ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5aa8105990 0x7f5aa80ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 -- 172.21.15.82:0/1188546454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aa8100750 msgr2=0x7f5aa80fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aa8100750 0x7f5aa80fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 -- 172.21.15.82:0/1188546454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0005e70 con 0x7f5aa8103070 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aacadb700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 0x7f5aa80fec40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800e9a0 tx=0x7f5a9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98009790 con 0x7f5aa8103070 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a98010430 con 0x7f5aa8103070 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa819fe90 con 0x7f5aa8103070 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98018680 con 0x7f5aa8103070 2023-07-14T19:28:19.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.439+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa81a0310 con 0x7f5aa8103070 2023-07-14T19:28:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.440+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa80fbca0 con 0x7f5aa8103070 2023-07-14T19:28:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.442+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5a980105a0 con 0x7f5aa8103070 2023-07-14T19:28:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.443+0000 7f5a9e7fc700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5a940a1150 0x7f5a940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.443+0000 7f5aad2dc700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5a940a1150 0x7f5a940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.443+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f5a98014070 con 0x7f5aa8103070 2023-07-14T19:28:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.443+0000 7f5aad2dc700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5a940a1150 0x7f5a940a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005950 tx=0x7f5aa400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.446+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5a980963c0 con 0x7f5aa8103070 2023-07-14T19:28:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.686+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7f5aa804e420 con 0x7f5aa8103070 2023-07-14T19:28:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.851+0000 7f5a9e7fc700 1 -- 172.21.15.82:0/1188546454 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) v1 ==== 117+0+0 (secure 0 0 0) 0x7f5a980260b0 con 0x7f5aa8103070 2023-07-14T19:28:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5a940a1150 msgr2=0x7f5a940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5a940a1150 0x7f5a940a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005950 tx=0x7f5aa400a300 comp rx=0 tx=0).stop 2023-07-14T19:28:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 msgr2=0x7f5aa80fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:19.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 0x7f5aa80fec40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800e9a0 tx=0x7f5a9800ecb0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 shutdown_connections 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5aa8105990 0x7f5aa80ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aa8100750 0x7f5aa80fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f5a940a1150 0x7f5a940a3600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 --2- 172.21.15.82:0/1188546454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aa8103070 0x7f5aa80fec40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 >> 172.21.15.82:0/1188546454 conn(0x7f5aa80fa130 msgr2=0x7f5aa81013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 shutdown_connections 2023-07-14T19:28:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:19.853+0000 7f5aaf540700 1 -- 172.21.15.82:0/1188546454 wait complete. 2023-07-14T19:28:19.867 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-14T19:28:19.879 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-14T19:28:19.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:19.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:19 smithi082 ceph-mon[127461]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T19:28:19.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:19 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1188546454' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T19:28:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.047+0000 7f29c7cbf700 1 -- 172.21.15.82:0/4067894771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c01006a0 msgr2=0x7f29c0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.047+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/4067894771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c01006a0 0x7f29c0106f60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b0a0 tx=0x7f29b800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- 172.21.15.82:0/4067894771 shutdown_connections 2023-07-14T19:28:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/4067894771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c01006a0 0x7f29c0106f60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/4067894771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c00feb20 0x7f29c00fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/4067894771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29c00ffd10 0x7f29c0100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- 172.21.15.82:0/4067894771 >> 172.21.15.82:0/4067894771 conn(0x7f29c00fa100 msgr2=0x7f29c00fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:20.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- 172.21.15.82:0/4067894771 shutdown_connections 2023-07-14T19:28:20.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- 172.21.15.82:0/4067894771 wait complete. 2023-07-14T19:28:20.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 Processor -- start 2023-07-14T19:28:20.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- start start 2023-07-14T19:28:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29c00feb20 0x7f29c01959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 0x7f29c0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c01006a0 0x7f29c019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c019d550 con 0x7f29c00ffd10 2023-07-14T19:28:20.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c019d690 con 0x7f29c01006a0 2023-07-14T19:28:20.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c7cbf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c019d820 con 0x7f29c00feb20 2023-07-14T19:28:20.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c525a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 0x7f29c0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c525a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 0x7f29c0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42052/0 (socket says 172.21.15.82:42052) 2023-07-14T19:28:20.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.048+0000 7f29c525a700 1 -- 172.21.15.82:0/2259374706 learned_addr learned my addr 172.21.15.82:0/2259374706 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:20.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c5a5b700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29c00feb20 0x7f29c01959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c625c700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c01006a0 0x7f29c019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c525a700 1 -- 172.21.15.82:0/2259374706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29c00feb20 msgr2=0x7f29c01959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c525a700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29c00feb20 0x7f29c01959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c525a700 1 -- 172.21.15.82:0/2259374706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c01006a0 msgr2=0x7f29c019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c525a700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c01006a0 0x7f29c019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c525a700 1 -- 172.21.15.82:0/2259374706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8009d00 con 0x7f29c00ffd10 2023-07-14T19:28:20.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c625c700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c01006a0 0x7f29c019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:20.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c525a700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 0x7f29c0195f10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f29b000d950 tx=0x7f29b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:20.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0009940 con 0x7f29c00ffd10 2023-07-14T19:28:20.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b0010460 con 0x7f29c00ffd10 2023-07-14T19:28:20.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c019db00 con 0x7f29c00ffd10 2023-07-14T19:28:20.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b000f620 con 0x7f29c00ffd10 2023-07-14T19:28:20.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.049+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c019e010 con 0x7f29c00ffd10 2023-07-14T19:28:20.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.050+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c010a300 con 0x7f29c00ffd10 2023-07-14T19:28:20.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.052+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f29b00108d0 con 0x7f29c00ffd10 2023-07-14T19:28:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.052+0000 7f29b6ffd700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f29ac0a11a0 0x7f29ac0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.053+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f29b00cc690 con 0x7f29c00ffd10 2023-07-14T19:28:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.053+0000 7f29c5a5b700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f29ac0a11a0 0x7f29ac0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.062 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.053+0000 7f29c5a5b700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f29ac0a11a0 0x7f29ac0a3650 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29c00ffb70 tx=0x7f29bc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:20.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.056+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f29b0095e00 con 0x7f29c00ffd10 2023-07-14T19:28:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:19 smithi133 ceph-mon[128894]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T19:28:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:19 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1188546454' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T19:28:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:19 smithi161 ceph-mon[129184]: pgmap v290: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T19:28:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:19 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1188546454' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2023-07-14T19:28:20.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.252+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29c019e5c0 con 0x7f29c00ffd10 2023-07-14T19:28:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.254+0000 7f29b6ffd700 1 -- 172.21.15.82:0/2259374706 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f29b0016090 con 0x7f29c00ffd10 2023-07-14T19:28:20.256 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:20.256 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f29ac0a11a0 msgr2=0x7f29ac0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f29ac0a11a0 0x7f29ac0a3650 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f29c00ffb70 tx=0x7f29bc007400 comp rx=0 tx=0).stop 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 msgr2=0x7f29c0195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 0x7f29c0195f10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f29b000d950 tx=0x7f29b000dc60 comp rx=0 tx=0).stop 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 shutdown_connections 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29c01006a0 0x7f29c019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29c00feb20 0x7f29c01959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f29ac0a11a0 0x7f29ac0a3650 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 --2- 172.21.15.82:0/2259374706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c00ffd10 0x7f29c0195f10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 >> 172.21.15.82:0/2259374706 conn(0x7f29c00fa100 msgr2=0x7f29c0068870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 shutdown_connections 2023-07-14T19:28:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.256+0000 7f29c7cbf700 1 -- 172.21.15.82:0/2259374706 wait complete. 2023-07-14T19:28:20.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:28:20.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:28:20.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 -- 172.21.15.82:0/241122394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 msgr2=0x7fb10c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 --2- 172.21.15.82:0/241122394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c0fef20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb108009990 tx=0x7fb108009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 -- 172.21.15.82:0/241122394 shutdown_connections 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 --2- 172.21.15.82:0/241122394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c100690 0x7fb10c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 --2- 172.21.15.82:0/241122394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c0fef20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 --2- 172.21.15.82:0/241122394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb10c0ffd00 0x7fb10c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 -- 172.21.15.82:0/241122394 >> 172.21.15.82:0/241122394 conn(0x7fb10c0fa130 msgr2=0x7fb10c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 -- 172.21.15.82:0/241122394 shutdown_connections 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 -- 172.21.15.82:0/241122394 wait complete. 2023-07-14T19:28:20.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 Processor -- start 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 -- start start 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.438+0000 7fb11331e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0ffd00 0x7fb10c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb10c100690 0x7fb10c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c19d560 con 0x7fb10c0feb10 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c19d6d0 con 0x7fb10c100690 2023-07-14T19:28:20.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c19d890 con 0x7fb10c0ffd00 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42062/0 (socket says 172.21.15.82:42062) 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 -- 172.21.15.82:0/872651135 learned_addr learned my addr 172.21.15.82:0/872651135 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1108b9700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0ffd00 0x7fb10c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1118bb700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb10c100690 0x7fb10c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 -- 172.21.15.82:0/872651135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0ffd00 msgr2=0x7fb10c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0ffd00 0x7fb10c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 -- 172.21.15.82:0/872651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb10c100690 msgr2=0x7fb10c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb10c100690 0x7fb10c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 -- 172.21.15.82:0/872651135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080095f0 con 0x7fb10c0feb10 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1108b9700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0ffd00 0x7fb10c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1110ba700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c068330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb1080098e0 tx=0x7fb108005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10801d070 con 0x7fb10c0feb10 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb108004ca0 con 0x7fb10c0feb10 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c19db10 con 0x7fb10c0feb10 2023-07-14T19:28:20.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108005510 con 0x7fb10c0feb10 2023-07-14T19:28:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.439+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c19df50 con 0x7fb10c0feb10 2023-07-14T19:28:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.440+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c105a90 con 0x7fb10c0feb10 2023-07-14T19:28:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.442+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb10800fc40 con 0x7fb10c0feb10 2023-07-14T19:28:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.442+0000 7fb1027fc700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb0f80a1150 0x7fb0f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.442+0000 7fb1108b9700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb0f80a1150 0x7fb0f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.442+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fb1080ce050 con 0x7fb10c0feb10 2023-07-14T19:28:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.443+0000 7fb1108b9700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb0f80a1150 0x7fb0f80a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0680a0 tx=0x7fb0fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:20.448 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.445+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb1080978f0 con 0x7fb10c0feb10 2023-07-14T19:28:20.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.633+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb10c10a420 con 0x7fb10c0feb10 2023-07-14T19:28:20.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.634+0000 7fb1027fc700 1 -- 172.21.15.82:0/872651135 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb10809af10 con 0x7fb10c0feb10 2023-07-14T19:28:20.635 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:28:20.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb0f80a1150 msgr2=0x7fb0f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb0f80a1150 0x7fb0f80a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0680a0 tx=0x7fb0fc009260 comp rx=0 tx=0).stop 2023-07-14T19:28:20.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 msgr2=0x7fb10c068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c068330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb1080098e0 tx=0x7fb108005e80 comp rx=0 tx=0).stop 2023-07-14T19:28:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 shutdown_connections 2023-07-14T19:28:20.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb10c100690 0x7fb10c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb0f80a1150 0x7fb0f80a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb10c0feb10 0x7fb10c068330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 --2- 172.21.15.82:0/872651135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0ffd00 0x7fb10c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 >> 172.21.15.82:0/872651135 conn(0x7fb10c0fa130 msgr2=0x7fb10c0fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 shutdown_connections 2023-07-14T19:28:20.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.636+0000 7fb11331e700 1 -- 172.21.15.82:0/872651135 wait complete. 2023-07-14T19:28:20.649 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:28:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 -- 172.21.15.82:0/2842049210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ffd70 msgr2=0x7f8e7c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 --2- 172.21.15.82:0/2842049210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ffd70 0x7f8e7c1001e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009990 tx=0x7f8e70009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 -- 172.21.15.82:0/2842049210 shutdown_connections 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 --2- 172.21.15.82:0/2842049210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e7c100720 0x7f8e7c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 --2- 172.21.15.82:0/2842049210 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c0ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 --2- 172.21.15.82:0/2842049210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ffd70 0x7f8e7c1001e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 -- 172.21.15.82:0/2842049210 >> 172.21.15.82:0/2842049210 conn(0x7f8e7c0f98e0 msgr2=0x7f8e7c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 -- 172.21.15.82:0/2842049210 shutdown_connections 2023-07-14T19:28:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 -- 172.21.15.82:0/2842049210 wait complete. 2023-07-14T19:28:20.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 Processor -- start 2023-07-14T19:28:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.772+0000 7f8e81585700 1 -- start start 2023-07-14T19:28:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c1948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e7c0ffd70 0x7f8e7c194e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c100720 0x7f8e7c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c19c450 con 0x7f8e7c0ff060 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c19c590 con 0x7f8e7c0ffd70 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c19c720 con 0x7f8e7c100720 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c1948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c1948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42076/0 (socket says 172.21.15.82:42076) 2023-07-14T19:28:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 -- 172.21.15.82:0/3660452685 learned_addr learned my addr 172.21.15.82:0/3660452685 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7a7fc700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e7c0ffd70 0x7f8e7c194e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7b7fe700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c100720 0x7f8e7c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 -- 172.21.15.82:0/3660452685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c100720 msgr2=0x7f8e7c19be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c100720 0x7f8e7c19be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 -- 172.21.15.82:0/3660452685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e7c0ffd70 msgr2=0x7f8e7c194e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e7c0ffd70 0x7f8e7c194e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 -- 172.21.15.82:0/3660452685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700095f0 con 0x7f8e7c0ff060 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e7affd700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c1948d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c970 tx=0x7f8e6c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c004220 con 0x7f8e7c0ff060 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6c007d60 con 0x7f8e7c0ff060 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c19ca00 con 0x7f8e7c0ff060 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c003d00 con 0x7f8e7c0ff060 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.773+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c19cf10 con 0x7f8e7c0ff060 2023-07-14T19:28:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.774+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c106110 con 0x7f8e7c0ff060 2023-07-14T19:28:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.775+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8e6c004380 con 0x7f8e7c0ff060 2023-07-14T19:28:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.776+0000 7f8e63fff700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e640a1150 0x7f8e640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.776+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f8e6c0cc830 con 0x7f8e7c0ff060 2023-07-14T19:28:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.776+0000 7f8e7a7fc700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e640a1150 0x7f8e640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.776+0000 7f8e7a7fc700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e640a1150 0x7f8e640a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000b5c0 tx=0x7f8e70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.779+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8e6c096040 con 0x7f8e7c0ff060 2023-07-14T19:28:20.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:20 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1188546454' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2023-07-14T19:28:20.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:20 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:28:20.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:20 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2259374706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:20.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:20 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/872651135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.975+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e7c02cb30 con 0x7f8e7c0ff060 2023-07-14T19:28:20.977 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.976+0000 7f8e63fff700 1 -- 172.21.15.82:0/3660452685 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e6c099660 con 0x7f8e7c0ff060 2023-07-14T19:28:20.978 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:28:20.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e640a1150 msgr2=0x7f8e640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.980 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e640a1150 0x7f8e640a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000b5c0 tx=0x7f8e70005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 msgr2=0x7f8e7c1948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c1948d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c970 tx=0x7f8e6c00cd30 comp rx=0 tx=0).stop 2023-07-14T19:28:20.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 shutdown_connections 2023-07-14T19:28:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e7c100720 0x7f8e7c19be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8e640a1150 0x7f8e640a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7c0ff060 0x7f8e7c1948d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.982 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 --2- 172.21.15.82:0/3660452685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e7c0ffd70 0x7f8e7c194e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 >> 172.21.15.82:0/3660452685 conn(0x7f8e7c0f98e0 msgr2=0x7f8e7c103250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 shutdown_connections 2023-07-14T19:28:20.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:20.978+0000 7f8e81585700 1 -- 172.21.15.82:0/3660452685 wait complete. 2023-07-14T19:28:20.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:28:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3747145445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 msgr2=0x7fe19c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3747145445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c107010 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe19400b0a0 tx=0x7fe19400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3747145445 shutdown_connections 2023-07-14T19:28:21.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3747145445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c107010 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3747145445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe19c0feb30 0x7fe19c0fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3747145445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0ffdc0 0x7fe19c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.116+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3747145445 >> 172.21.15.82:0/3747145445 conn(0x7fe19c0fa150 msgr2=0x7fe19c0fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:21.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3747145445 shutdown_connections 2023-07-14T19:28:21.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3747145445 wait complete. 2023-07-14T19:28:21.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 Processor -- start 2023-07-14T19:28:21.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 -- start start 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0feb30 0x7fe19c0694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe19c0ffdc0 0x7fe19c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c19d510 con 0x7fe19c100770 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c19d680 con 0x7fe19c0ffdc0 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.117+0000 7fe1a08d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c19d840 con 0x7fe19c0feb30 2023-07-14T19:28:21.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42094/0 (socket says 172.21.15.82:42094) 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 -- 172.21.15.82:0/3829338153 learned_addr learned my addr 172.21.15.82:0/3829338153 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19a59c700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0feb30 0x7fe19c0694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe199d9b700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe19c0ffdc0 0x7fe19c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 -- 172.21.15.82:0/3829338153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0feb30 msgr2=0x7fe19c0694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0feb30 0x7fe19c0694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 -- 172.21.15.82:0/3829338153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe19c0ffdc0 msgr2=0x7fe19c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe19c0ffdc0 0x7fe19c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 -- 172.21.15.82:0/3829338153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194009d00 con 0x7fe19c100770 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19a59c700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0feb30 0x7fe19c0694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe19ad9d700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c19cd70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe1940062a0 tx=0x7fe194004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400e040 con 0x7fe19c100770 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19c19dac0 con 0x7fe19c100770 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe19401db50 con 0x7fe19c100770 2023-07-14T19:28:21.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19401bc40 con 0x7fe19c100770 2023-07-14T19:28:21.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.118+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c1947d0 con 0x7fe19c100770 2023-07-14T19:28:21.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.119+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c10a300 con 0x7fe19c100770 2023-07-14T19:28:21.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.121+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe19401d690 con 0x7fe19c100770 2023-07-14T19:28:21.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.121+0000 7fe18b7fe700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe1840a1150 0x7fe1840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.121+0000 7fe19a59c700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe1840a1150 0x7fe1840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.121+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fe1940ceb30 con 0x7fe19c100770 2023-07-14T19:28:21.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.122+0000 7fe19a59c700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe1840a1150 0x7fe1840a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe18c005fd0 tx=0x7fe18c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:21.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.124+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe1940983d0 con 0x7fe19c100770 2023-07-14T19:28:21.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:20 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1188546454' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2023-07-14T19:28:21.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:20 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:28:21.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:20 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2259374706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:21.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:20 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/872651135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:21.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:20 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1188546454' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2023-07-14T19:28:21.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:20 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:28:21.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:20 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2259374706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:21.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:20 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/872651135' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:21.313 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.312+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe19c194e70 con 0x7fe19c100770 2023-07-14T19:28:21.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.312+0000 7fe18b7fe700 1 -- 172.21.15.82:0/3829338153 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe19409b9f0 con 0x7fe19c100770 2023-07-14T19:28:21.314 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:28:21.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe1840a1150 msgr2=0x7fe1840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe1840a1150 0x7fe1840a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe18c005fd0 tx=0x7fe18c005e30 comp rx=0 tx=0).stop 2023-07-14T19:28:21.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 msgr2=0x7fe19c19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c19cd70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe1940062a0 tx=0x7fe194004db0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 shutdown_connections 2023-07-14T19:28:21.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe1840a1150 0x7fe1840a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe19c100770 0x7fe19c19cd70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.314+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe19c0feb30 0x7fe19c0694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.315+0000 7fe1a08d6700 1 --2- 172.21.15.82:0/3829338153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe19c0ffdc0 0x7fe19c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.315+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 >> 172.21.15.82:0/3829338153 conn(0x7fe19c0fa150 msgr2=0x7fe19c0fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:21.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.315+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 shutdown_connections 2023-07-14T19:28:21.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.315+0000 7fe1a08d6700 1 -- 172.21.15.82:0/3829338153 wait complete. 2023-07-14T19:28:21.328 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:21.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 -- 172.21.15.82:0/3657422220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05180ffcf0 msgr2=0x7f0518100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3657422220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05180ffcf0 0x7f0518100140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0514009990 tx=0x7f0514009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.450 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 -- 172.21.15.82:0/3657422220 shutdown_connections 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3657422220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0518100680 0x7f0518106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3657422220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180feb00 0x7f05180fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3657422220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05180ffcf0 0x7f0518100140 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 -- 172.21.15.82:0/3657422220 >> 172.21.15.82:0/3657422220 conn(0x7f05180fa100 msgr2=0x7f05180fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 -- 172.21.15.82:0/3657422220 shutdown_connections 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 -- 172.21.15.82:0/3657422220 wait complete. 2023-07-14T19:28:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 Processor -- start 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.447+0000 7f051f6a0700 1 -- start start 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05180feb00 0x7f0518195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180ffcf0 0x7f0518195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 0x7f051819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051819d450 con 0x7f0518100680 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051819d590 con 0x7f05180ffcf0 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051819d720 con 0x7f05180feb00 2023-07-14T19:28:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 0x7f051819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 0x7f051819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42108/0 (socket says 172.21.15.82:42108) 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 -- 172.21.15.82:0/3462374722 learned_addr learned my addr 172.21.15.82:0/3462374722 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051cc3b700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180ffcf0 0x7f0518195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051d43c700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05180feb00 0x7f0518195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 -- 172.21.15.82:0/3462374722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05180feb00 msgr2=0x7f0518195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05180feb00 0x7f0518195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 -- 172.21.15.82:0/3462374722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180ffcf0 msgr2=0x7f0518195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180ffcf0 0x7f0518195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 -- 172.21.15.82:0/3462374722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140095f0 con 0x7f0518100680 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051cc3b700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180ffcf0 0x7f0518195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051dc3d700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 0x7f051819ce80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f051000ada0 tx=0x7f051000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051000f070 con 0x7f0518100680 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0510007b30 con 0x7f0518100680 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051819da00 con 0x7f0518100680 2023-07-14T19:28:21.454 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510005440 con 0x7f0518100680 2023-07-14T19:28:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.448+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051819dec0 con 0x7f0518100680 2023-07-14T19:28:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.449+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051810a2d0 con 0x7f0518100680 2023-07-14T19:28:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.450+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0510004ad0 con 0x7f0518100680 2023-07-14T19:28:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.451+0000 7f050a7fc700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f05040a11a0 0x7f05040a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.451+0000 7f051d43c700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f05040a11a0 0x7f05040a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.451+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f05100cbf30 con 0x7f0518100680 2023-07-14T19:28:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.451+0000 7f051d43c700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f05040a11a0 0x7f05040a3650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f050c005950 tx=0x7f050c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:21.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.455+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f05100957d0 con 0x7f0518100680 2023-07-14T19:28:21.647 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.646+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f051819e4d0 con 0x7f0518100680 2023-07-14T19:28:21.648 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.647+0000 7f050a7fc700 1 -- 172.21.15.82:0/3462374722 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f0510098df0 con 0x7f0518100680 2023-07-14T19:28:21.649 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:21.649 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f05040a11a0 msgr2=0x7f05040a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f05040a11a0 0x7f05040a3650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f050c005950 tx=0x7f050c0058e0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 msgr2=0x7f051819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 0x7f051819ce80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f051000ada0 tx=0x7f051000c330 comp rx=0 tx=0).stop 2023-07-14T19:28:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 shutdown_connections 2023-07-14T19:28:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f05040a11a0 0x7f05040a3650 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0518100680 0x7f051819ce80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05180feb00 0x7f0518195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 --2- 172.21.15.82:0/3462374722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05180ffcf0 0x7f0518195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 >> 172.21.15.82:0/3462374722 conn(0x7f05180fa100 msgr2=0x7f0518067820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.649+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 shutdown_connections 2023-07-14T19:28:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.650+0000 7f051f6a0700 1 -- 172.21.15.82:0/3462374722 wait complete. 2023-07-14T19:28:21.663 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:28:21.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:28:21.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 -- 172.21.15.82:0/2770837235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd4100e40 msgr2=0x7f1fd4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/2770837235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd4100e40 0x7f1fd4101290 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009990 tx=0x7f1fc8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 -- 172.21.15.82:0/2770837235 shutdown_connections 2023-07-14T19:28:21.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/2770837235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd41017d0 0x7f1fd4108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/2770837235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd40ffc50 0x7f1fd4100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/2770837235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd4100e40 0x7f1fd4101290 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 -- 172.21.15.82:0/2770837235 >> 172.21.15.82:0/2770837235 conn(0x7f1fd40fb290 msgr2=0x7f1fd40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 -- 172.21.15.82:0/2770837235 shutdown_connections 2023-07-14T19:28:21.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.787+0000 7f1fd8efd700 1 -- 172.21.15.82:0/2770837235 wait complete. 2023-07-14T19:28:21.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 Processor -- start 2023-07-14T19:28:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 -- start start 2023-07-14T19:28:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd40ffc50 0x7f1fd4195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd4100e40 0x7f1fd4195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 0x7f1fd419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd419d4f0 con 0x7f1fd41017d0 2023-07-14T19:28:21.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd419d630 con 0x7f1fd40ffc50 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd8efd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd419d7c0 con 0x7f1fd4100e40 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 0x7f1fd419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 0x7f1fd419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42122/0 (socket says 172.21.15.82:42122) 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 -- 172.21.15.82:0/3752911541 learned_addr learned my addr 172.21.15.82:0/3752911541 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd259c700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd40ffc50 0x7f1fd4195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd1d9b700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd4100e40 0x7f1fd4195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 -- 172.21.15.82:0/3752911541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd4100e40 msgr2=0x7f1fd4195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd4100e40 0x7f1fd4195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 -- 172.21.15.82:0/3752911541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd40ffc50 msgr2=0x7f1fd4195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd40ffc50 0x7f1fd4195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 -- 172.21.15.82:0/3752911541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80095f0 con 0x7f1fd41017d0 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd259c700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd40ffc50 0x7f1fd4195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fd2d9d700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 0x7f1fd419cf20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc007ee0 tx=0x7f1fcc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.788+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017070 con 0x7f1fd41017d0 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.789+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd419daa0 con 0x7f1fd41017d0 2023-07-14T19:28:21.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.789+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fcc00f040 con 0x7f1fd41017d0 2023-07-14T19:28:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.789+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc005740 con 0x7f1fd41017d0 2023-07-14T19:28:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.789+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd419df60 con 0x7f1fd41017d0 2023-07-14T19:28:21.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.789+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd4107190 con 0x7f1fd41017d0 2023-07-14T19:28:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.791+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1fcc0058a0 con 0x7f1fd41017d0 2023-07-14T19:28:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.791+0000 7f1fc37fe700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1fbc0a1150 0x7f1fbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.791+0000 7f1fd259c700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1fbc0a1150 0x7f1fbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.791+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f1fcc0cc6f0 con 0x7f1fd41017d0 2023-07-14T19:28:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.792+0000 7f1fd259c700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1fbc0a1150 0x7f1fbc0a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4100ca0 tx=0x7f1fc400a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:21.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.795+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1fcc095f00 con 0x7f1fd41017d0 2023-07-14T19:28:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:21 smithi082 ceph-mon[127461]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T19:28:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:21 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3660452685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:21 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3829338153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:21.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:21 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3462374722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:21.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.983+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fd402ca70 con 0x7f1fd41017d0 2023-07-14T19:28:21.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.983+0000 7f1fc37fe700 1 -- 172.21.15.82:0/3752911541 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fcc099520 con 0x7f1fd41017d0 2023-07-14T19:28:21.985 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:28:21.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1fbc0a1150 msgr2=0x7f1fbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1fbc0a1150 0x7f1fbc0a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4100ca0 tx=0x7f1fc400a680 comp rx=0 tx=0).stop 2023-07-14T19:28:21.987 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 msgr2=0x7f1fd419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 0x7f1fd419cf20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc007ee0 tx=0x7f1fcc00d3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 shutdown_connections 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f1fbc0a1150 0x7f1fbc0a3600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41017d0 0x7f1fd419cf20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd40ffc50 0x7f1fd4195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 --2- 172.21.15.82:0/3752911541 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd4100e40 0x7f1fd4195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 >> 172.21.15.82:0/3752911541 conn(0x7f1fd40fb290 msgr2=0x7f1fd41042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 shutdown_connections 2023-07-14T19:28:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:21.986+0000 7f1fd8efd700 1 -- 172.21.15.82:0/3752911541 wait complete. 2023-07-14T19:28:21.999 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:28:22.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 -- 172.21.15.82:0/1364361670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd334100770 msgr2=0x7fd334107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 --2- 172.21.15.82:0/1364361670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd334100770 0x7fd334107010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00b0a0 tx=0x7fd32c00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 -- 172.21.15.82:0/1364361670 shutdown_connections 2023-07-14T19:28:22.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 --2- 172.21.15.82:0/1364361670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd334100770 0x7fd334107010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.123 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 --2- 172.21.15.82:0/1364361670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3340feb30 0x7fd3340fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 --2- 172.21.15.82:0/1364361670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3340ffdc0 0x7fd334100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 -- 172.21.15.82:0/1364361670 >> 172.21.15.82:0/1364361670 conn(0x7fd3340fa150 msgr2=0x7fd3340fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 -- 172.21.15.82:0/1364361670 shutdown_connections 2023-07-14T19:28:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 -- 172.21.15.82:0/1364361670 wait complete. 2023-07-14T19:28:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 Processor -- start 2023-07-14T19:28:22.124 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.121+0000 7fd33a127700 1 -- start start 2023-07-14T19:28:22.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3340feb30 0x7fd3340694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 0x7fd33419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd334100770 0x7fd33419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33419d5d0 con 0x7fd3340ffdc0 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33419d740 con 0x7fd3340feb30 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33419d900 con 0x7fd334100770 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 0x7fd33419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 0x7fd33419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42134/0 (socket says 172.21.15.82:42134) 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 -- 172.21.15.82:0/1444030487 learned_addr learned my addr 172.21.15.82:0/1444030487 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd3337fe700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3340feb30 0x7fd3340694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd333fff700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd334100770 0x7fd33419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 -- 172.21.15.82:0/1444030487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd334100770 msgr2=0x7fd33419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd334100770 0x7fd33419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 -- 172.21.15.82:0/1444030487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3340feb30 msgr2=0x7fd3340694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3340feb30 0x7fd3340694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 -- 172.21.15.82:0/1444030487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c009d00 con 0x7fd3340ffdc0 2023-07-14T19:28:22.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd333fff700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd334100770 0x7fd33419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd332ffd700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 0x7fd33419a7b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d950 tx=0x7fd32400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324009940 con 0x7fd3340ffdc0 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd324010460 con 0x7fd3340ffdc0 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33419dbe0 con 0x7fd3340ffdc0 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32400f620 con 0x7fd3340ffdc0 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.122+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33419e0f0 con 0x7fd3340ffdc0 2023-07-14T19:28:22.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.123+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33410a350 con 0x7fd3340ffdc0 2023-07-14T19:28:22.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.124+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd3240108d0 con 0x7fd3340ffdc0 2023-07-14T19:28:22.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.125+0000 7fd330ff9700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd31c0a1150 0x7fd31c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.125+0000 7fd3337fe700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd31c0a1150 0x7fd31c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.125+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fd3240cc710 con 0x7fd3340ffdc0 2023-07-14T19:28:22.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.125+0000 7fd3337fe700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd31c0a1150 0x7fd31c0a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd328005950 tx=0x7fd32800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:22.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.128+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd324095e80 con 0x7fd3340ffdc0 2023-07-14T19:28:22.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:21 smithi133 ceph-mon[128894]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T19:28:22.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:21 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3660452685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:22.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:21 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3829338153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:22.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:21 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3462374722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:22.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:21 smithi161 ceph-mon[129184]: pgmap v291: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-07-14T19:28:22.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:21 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3660452685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:22.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:21 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3829338153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:22.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:21 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3462374722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:22.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.316+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd334105a60 con 0x7fd3340ffdc0 2023-07-14T19:28:22.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.317+0000 7fd330ff9700 1 -- 172.21.15.82:0/1444030487 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd324016020 con 0x7fd3340ffdc0 2023-07-14T19:28:22.319 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:28:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd31c0a1150 msgr2=0x7fd31c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd31c0a1150 0x7fd31c0a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd328005950 tx=0x7fd32800a300 comp rx=0 tx=0).stop 2023-07-14T19:28:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 msgr2=0x7fd33419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 0x7fd33419a7b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd32400d950 tx=0x7fd32400dc60 comp rx=0 tx=0).stop 2023-07-14T19:28:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 shutdown_connections 2023-07-14T19:28:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd334100770 0x7fd33419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3340feb30 0x7fd3340694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd31c0a1150 0x7fd31c0a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 --2- 172.21.15.82:0/1444030487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3340ffdc0 0x7fd33419a7b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 >> 172.21.15.82:0/1444030487 conn(0x7fd3340fa150 msgr2=0x7fd3340fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 shutdown_connections 2023-07-14T19:28:22.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.319+0000 7fd33a127700 1 -- 172.21.15.82:0/1444030487 wait complete. 2023-07-14T19:28:22.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:28:22.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/3543534519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0ffd00 msgr2=0x7ff8cc100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/3543534519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0ffd00 0x7ff8cc100150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009990 tx=0x7ff8bc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/3543534519 shutdown_connections 2023-07-14T19:28:22.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/3543534519 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 0x7ff8cc106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/3543534519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/3543534519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0ffd00 0x7ff8cc100150 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/3543534519 >> 172.21.15.82:0/3543534519 conn(0x7ff8cc0fa130 msgr2=0x7ff8cc0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:22.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/3543534519 shutdown_connections 2023-07-14T19:28:22.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.454+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/3543534519 wait complete. 2023-07-14T19:28:22.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 Processor -- start 2023-07-14T19:28:22.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 -- start start 2023-07-14T19:28:22.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc0ffd00 0x7ff8cc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 0x7ff8cc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc19d510 con 0x7ff8cc0feb10 2023-07-14T19:28:22.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc19d680 con 0x7ff8cc0ffd00 2023-07-14T19:28:22.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8d1ca2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc19d840 con 0x7ff8cc100690 2023-07-14T19:28:22.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42144/0 (socket says 172.21.15.82:42144) 2023-07-14T19:28:22.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8cb7fe700 1 -- 172.21.15.82:0/820690674 learned_addr learned my addr 172.21.15.82:0/820690674 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:22.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8caffd700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc0ffd00 0x7ff8cc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8cbfff700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 0x7ff8cc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8cb7fe700 1 -- 172.21.15.82:0/820690674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 msgr2=0x7ff8cc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.455+0000 7ff8cb7fe700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 0x7ff8cc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8cb7fe700 1 -- 172.21.15.82:0/820690674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc0ffd00 msgr2=0x7ff8cc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8cb7fe700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc0ffd00 0x7ff8cc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8cb7fe700 1 -- 172.21.15.82:0/820690674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0095f0 con 0x7ff8cc0feb10 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8cbfff700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 0x7ff8cc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8cb7fe700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc068330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400e970 tx=0x7ff8b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4009790 con 0x7ff8cc0feb10 2023-07-14T19:28:22.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b40044e0 con 0x7ff8cc0feb10 2023-07-14T19:28:22.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc19db20 con 0x7ff8cc0feb10 2023-07-14T19:28:22.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003e30 con 0x7ff8cc0feb10 2023-07-14T19:28:22.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.456+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc19e030 con 0x7ff8cc0feb10 2023-07-14T19:28:22.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.457+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc105ac0 con 0x7ff8cc0feb10 2023-07-14T19:28:22.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.458+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff8b4003f90 con 0x7ff8cc0feb10 2023-07-14T19:28:22.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.459+0000 7ff8c8ff9700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff8b80a1150 0x7ff8b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.459+0000 7ff8caffd700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff8b80a1150 0x7ff8b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.459+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7ff8b4014070 con 0x7ff8cc0feb10 2023-07-14T19:28:22.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.459+0000 7ff8caffd700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff8b80a1150 0x7ff8b80a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00b2e0 tx=0x7ff8bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:22.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.462+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7ff8b40964a0 con 0x7ff8cc0feb10 2023-07-14T19:28:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.652+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff8cc02ca70 con 0x7ff8cc0feb10 2023-07-14T19:28:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.652+0000 7ff8c8ff9700 1 -- 172.21.15.82:0/820690674 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff8b400f3c0 con 0x7ff8cc0feb10 2023-07-14T19:28:22.654 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:28:22.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff8b80a1150 msgr2=0x7ff8b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff8b80a1150 0x7ff8b80a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00b2e0 tx=0x7ff8bc005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 msgr2=0x7ff8cc068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc068330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400e970 tx=0x7ff8b400ed30 comp rx=0 tx=0).stop 2023-07-14T19:28:22.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 shutdown_connections 2023-07-14T19:28:22.656 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8cc100690 0x7ff8cc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7ff8b80a1150 0x7ff8b80a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8cc0feb10 0x7ff8cc068330 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 --2- 172.21.15.82:0/820690674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8cc0ffd00 0x7ff8cc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 >> 172.21.15.82:0/820690674 conn(0x7ff8cc0fa130 msgr2=0x7ff8cc0fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:22.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 shutdown_connections 2023-07-14T19:28:22.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.654+0000 7ff8d1ca2700 1 -- 172.21.15.82:0/820690674 wait complete. 2023-07-14T19:28:22.668 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-14T19:28:22.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:28:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 -- 172.21.15.82:0/1198150447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f8100180 msgr2=0x7fe7f8100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1198150447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f8100180 0x7fe7f8100590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009990 tx=0x7fe7e8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 -- 172.21.15.82:0/1198150447 shutdown_connections 2023-07-14T19:28:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1198150447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1198150447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f8100180 0x7fe7f8100590 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1198150447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f80fed40 0x7fe7f80ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 -- 172.21.15.82:0/1198150447 >> 172.21.15.82:0/1198150447 conn(0x7fe7f80f98a0 msgr2=0x7fe7f80fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 -- 172.21.15.82:0/1198150447 shutdown_connections 2023-07-14T19:28:22.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.787+0000 7fe7fe550700 1 -- 172.21.15.82:0/1198150447 wait complete. 2023-07-14T19:28:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 Processor -- start 2023-07-14T19:28:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 -- start start 2023-07-14T19:28:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f80fed40 0x7fe7f80fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f8100180 0x7fe7f819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819f6a0 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819f810 con 0x7fe7f80fed40 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7fe550700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819f9d0 con 0x7fe7f8100180 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7f77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7f77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42160/0 (socket says 172.21.15.82:42160) 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7f77fe700 1 -- 172.21.15.82:0/1643966330 learned_addr learned my addr 172.21.15.82:0/1643966330 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:22.793 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.788+0000 7fe7f7fff700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f80fed40 0x7fe7f80fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7fcaed700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f8100180 0x7fe7f819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f77fe700 1 -- 172.21.15.82:0/1643966330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f8100180 msgr2=0x7fe7f819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f77fe700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f8100180 0x7fe7f819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f77fe700 1 -- 172.21.15.82:0/1643966330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f80fed40 msgr2=0x7fe7f80fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:22.794 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f77fe700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f80fed40 0x7fe7f80fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:22.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f77fe700 1 -- 172.21.15.82:0/1643966330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80095f0 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f77fe700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f819caa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00d950 tx=0x7fe7ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:22.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec00f890 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ec010460 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f819fcb0 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.795 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec00e610 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.789+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f81a01c0 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.796 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.790+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f810a320 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.797 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.791+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe7ec0108d0 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.792+0000 7fe7f57fa700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7e00a1220 0x7fe7e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:22.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.792+0000 7fe7f7fff700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7e00a1220 0x7fe7e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:22.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.792+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fe7ec0cc580 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.792+0000 7fe7f7fff700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7e00a1220 0x7fe7e00a36d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800b5c0 tx=0x7fe7e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:22.798 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:22.795+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe7ec095d90 con 0x7fe7f80ff6f0 2023-07-14T19:28:22.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:22 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3752911541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:22.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:22 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1444030487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:22.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:22 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/820690674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.029+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7f8105a10 con 0x7fe7f80ff6f0 2023-07-14T19:28:23.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.030+0000 7fe7f57fa700 1 -- 172.21.15.82:0/1643966330 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fe7ec020090 con 0x7fe7f80ff6f0 2023-07-14T19:28:23.032 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:23.032 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:28:19.848679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7e00a1220 msgr2=0x7fe7e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7e00a1220 0x7fe7e00a36d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800b5c0 tx=0x7fe7e8005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 msgr2=0x7fe7f819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f819caa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00d950 tx=0x7fe7ec00dc60 comp rx=0 tx=0).stop 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 shutdown_connections 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7f8100180 0x7fe7f819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f80fed40 0x7fe7f80fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe7e00a1220 0x7fe7e00a36d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 --2- 172.21.15.82:0/1643966330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f80ff6f0 0x7fe7f819caa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 >> 172.21.15.82:0/1643966330 conn(0x7fe7f80f98a0 msgr2=0x7fe7f80fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 shutdown_connections 2023-07-14T19:28:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.032+0000 7fe7fe550700 1 -- 172.21.15.82:0/1643966330 wait complete. 2023-07-14T19:28:23.035 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 23 2023-07-14T19:28:23.046 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-14T19:28:23.046 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-14T19:28:23.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T19:28:23.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 -- 172.21.15.82:0/1674571994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c0ffc20 msgr2=0x7f0b3c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 --2- 172.21.15.82:0/1674571994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c0ffc20 0x7f0b3c100030 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009990 tx=0x7f0b38009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 -- 172.21.15.82:0/1674571994 shutdown_connections 2023-07-14T19:28:23.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 --2- 172.21.15.82:0/1674571994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 --2- 172.21.15.82:0/1674571994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c0ffc20 0x7f0b3c100030 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.169 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 --2- 172.21.15.82:0/1674571994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b3c100e10 0x7f0b3c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 -- 172.21.15.82:0/1674571994 >> 172.21.15.82:0/1674571994 conn(0x7f0b3c0fb260 msgr2=0x7f0b3c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 -- 172.21.15.82:0/1674571994 shutdown_connections 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 -- 172.21.15.82:0/1674571994 wait complete. 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 Processor -- start 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.166+0000 7f0b43465700 1 -- start start 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0ffc20 0x7f0b3c1958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.170 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b3c100e10 0x7f0b3c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c19d450 con 0x7f0b3c1017a0 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c19d590 con 0x7f0b3c0ffc20 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c19d720 con 0x7f0b3c100e10 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42176/0 (socket says 172.21.15.82:42176) 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 -- 172.21.15.82:0/1045126042 learned_addr learned my addr 172.21.15.82:0/1045126042 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:23.171 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41201700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0ffc20 0x7f0b3c1958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b40a00700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b3c100e10 0x7f0b3c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 -- 172.21.15.82:0/1045126042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b3c100e10 msgr2=0x7f0b3c195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b3c100e10 0x7f0b3c195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 -- 172.21.15.82:0/1045126042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0ffc20 msgr2=0x7f0b3c1958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0ffc20 0x7f0b3c1958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 -- 172.21.15.82:0/1045126042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b380095f0 con 0x7f0b3c1017a0 2023-07-14T19:28:23.172 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b41a02700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c19ce80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400ef30 tx=0x7f0b3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:23.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34018070 con 0x7f0b3c1017a0 2023-07-14T19:28:23.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3400f040 con 0x7f0b3c1017a0 2023-07-14T19:28:23.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c19d9a0 con 0x7f0b3c1017a0 2023-07-14T19:28:23.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400cdb0 con 0x7f0b3c1017a0 2023-07-14T19:28:23.173 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.167+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c19de60 con 0x7f0b3c1017a0 2023-07-14T19:28:23.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.168+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c106b00 con 0x7f0b3c1017a0 2023-07-14T19:28:23.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.169+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0b34003a00 con 0x7f0b3c1017a0 2023-07-14T19:28:23.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.170+0000 7f0b327fc700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0b280a11a0 0x7f0b280a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.170+0000 7f0b41201700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0b280a11a0 0x7f0b280a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.174 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.170+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f0b340cc520 con 0x7f0b3c1017a0 2023-07-14T19:28:23.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.171+0000 7f0b41201700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0b280a11a0 0x7f0b280a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0b38006010 tx=0x7f0b38005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:23.175 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.174+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0b34095d20 con 0x7f0b3c1017a0 2023-07-14T19:28:23.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:22 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3752911541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:22 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1444030487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:22 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/820690674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:22 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3752911541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:22 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1444030487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:22 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/820690674' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:23.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.359+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b3c060a80 con 0x7f0b280a11a0 2023-07-14T19:28:23.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.369+0000 7f0b327fc700 1 -- 172.21.15.82:0/1045126042 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344000 (secure 0 0 0) 0x7f0b3c060a80 con 0x7f0b280a11a0 2023-07-14T19:28:23.374 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.372+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0b280a11a0 msgr2=0x7f0b280a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.372+0000 7f0b43465700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0b280a11a0 0x7f0b280a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0b38006010 tx=0x7f0b38005960 comp rx=0 tx=0).stop 2023-07-14T19:28:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 msgr2=0x7f0b3c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c19ce80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400ef30 tx=0x7f0b3400c3a0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 shutdown_connections 2023-07-14T19:28:23.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f0b280a11a0 0x7f0b280a3650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3c1017a0 0x7f0b3c19ce80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0ffc20 0x7f0b3c1958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 --2- 172.21.15.82:0/1045126042 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b3c100e10 0x7f0b3c195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 >> 172.21.15.82:0/1045126042 conn(0x7f0b3c0fb260 msgr2=0x7f0b3c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 shutdown_connections 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.373+0000 7f0b43465700 1 -- 172.21.15.82:0/1045126042 wait complete. 2023-07-14T19:28:23.377 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:28:23.386 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-07-14T19:28:22.738141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651866,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":641,"num_read_kb":437,"num_write":340,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":777,"ondisk_log_size":777,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335172,"kb_used_data":10980,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606652,"statfs":{"total":1151940427776,"available":1151597211648,"internally_reserved":0,"allocated":11243520,"data_stored":7137562,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":47,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766460+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:28:04.766460+0000","last_peered":"2023-07-14T19:28:04.766460+0000","last_clean":"2023-07-14T19:28:04.766460+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:28:04.766460+0000","last_undegraded":"2023-07-14T19:28:04.766460+0000","last_fullsized":"2023-07-14T19:28:04.766460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:41.471339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769021+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:28:04.769021+0000","last_peered":"2023-07-14T19:28:04.769021+0000","last_clean":"2023-07-14T19:28:04.769021+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:28:04.769021+0000","last_undegraded":"2023-07-14T19:28:04.769021+0000","last_fullsized":"2023-07-14T19:28:04.769021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:01.099918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765502+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:28:04.765502+0000","last_peered":"2023-07-14T19:28:04.765502+0000","last_clean":"2023-07-14T19:28:04.765502+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:28:04.765502+0000","last_undegraded":"2023-07-14T19:28:04.765502+0000","last_fullsized":"2023-07-14T19:28:04.765502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:33:44.231618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.761968+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:28:04.761968+0000","last_peered":"2023-07-14T19:28:04.761968+0000","last_clean":"2023-07-14T19:28:04.761968+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:28:04.761968+0000","last_undegraded":"2023-07-14T19:28:04.761968+0000","last_fullsized":"2023-07-14T19:28:04.761968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:38:02.561344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.757252+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:28:04.757252+0000","last_peered":"2023-07-14T19:28:04.757252+0000","last_clean":"2023-07-14T19:28:04.757252+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:28:04.757252+0000","last_undegraded":"2023-07-14T19:28:04.757252+0000","last_fullsized":"2023-07-14T19:28:04.757252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:36:44.500967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759197+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:28:04.759197+0000","last_peered":"2023-07-14T19:28:04.759197+0000","last_clean":"2023-07-14T19:28:04.759197+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:28:04.759197+0000","last_undegraded":"2023-07-14T19:28:04.759197+0000","last_fullsized":"2023-07-14T19:28:04.759197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.697906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764042+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:28:04.764042+0000","last_peered":"2023-07-14T19:28:04.764042+0000","last_clean":"2023-07-14T19:28:04.764042+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:28:04.764042+0000","last_undegraded":"2023-07-14T19:28:04.764042+0000","last_fullsized":"2023-07-14T19:28:04.764042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768382+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:28:04.768382+0000","last_peered":"2023-07-14T19:28:04.768382+0000","last_clean":"2023-07-14T19:28:04.768382+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:28:04.768382+0000","last_undegraded":"2023-07-14T19:28:04.768382+0000","last_fullsized":"2023-07-14T19:28:04.768382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:11.270082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765502+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:28:04.765502+0000","last_peered":"2023-07-14T19:28:04.765502+0000","last_clean":"2023-07-14T19:28:04.765502+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:28:04.765502+0000","last_undegraded":"2023-07-14T19:28:04.765502+0000","last_fullsized":"2023-07-14T19:28:04.765502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:57:14.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758142+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:28:04.758142+0000","last_peered":"2023-07-14T19:28:04.758142+0000","last_clean":"2023-07-14T19:28:04.758142+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:28:04.758142+0000","last_undegraded":"2023-07-14T19:28:04.758142+0000","last_fullsized":"2023-07-14T19:28:04.758142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:09.380546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766790+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:28:04.766790+0000","last_peered":"2023-07-14T19:28:04.766790+0000","last_clean":"2023-07-14T19:28:04.766790+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:28:04.766790+0000","last_undegraded":"2023-07-14T19:28:04.766790+0000","last_fullsized":"2023-07-14T19:28:04.766790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:24.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763507+0000","last_change":"2023-07-14T19:27:05.797079+0000","last_active":"2023-07-14T19:28:04.763507+0000","last_peered":"2023-07-14T19:28:04.763507+0000","last_clean":"2023-07-14T19:28:04.763507+0000","last_became_active":"2023-07-14T19:27:05.796765+0000","last_became_peered":"2023-07-14T19:27:05.796765+0000","last_unstale":"2023-07-14T19:28:04.763507+0000","last_undegraded":"2023-07-14T19:28:04.763507+0000","last_fullsized":"2023-07-14T19:28:04.763507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:18.699215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760253+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:28:04.760253+0000","last_peered":"2023-07-14T19:28:04.760253+0000","last_clean":"2023-07-14T19:28:04.760253+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:28:04.760253+0000","last_undegraded":"2023-07-14T19:28:04.760253+0000","last_fullsized":"2023-07-14T19:28:04.760253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:24.688894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768895+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:28:04.768895+0000","last_peered":"2023-07-14T19:28:04.768895+0000","last_clean":"2023-07-14T19:28:04.768895+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:28:04.768895+0000","last_undegraded":"2023-07-14T19:28:04.768895+0000","last_fullsized":"2023-07-14T19:28:04.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:11:02.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768383+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:28:04.768383+0000","last_peered":"2023-07-14T19:28:04.768383+0000","last_clean":"2023-07-14T19:28:04.768383+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:28:04.768383+0000","last_undegraded":"2023-07-14T19:28:04.768383+0000","last_fullsized":"2023-07-14T19:28:04.768383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:29.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764585+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:28:04.764585+0000","last_peered":"2023-07-14T19:28:04.764585+0000","last_clean":"2023-07-14T19:28:04.764585+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:28:04.764585+0000","last_undegraded":"2023-07-14T19:28:04.764585+0000","last_fullsized":"2023-07-14T19:28:04.764585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:14.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766558+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:28:04.766558+0000","last_peered":"2023-07-14T19:28:04.766558+0000","last_clean":"2023-07-14T19:28:04.766558+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:28:04.766558+0000","last_undegraded":"2023-07-14T19:28:04.766558+0000","last_fullsized":"2023-07-14T19:28:04.766558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:47.623791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762716+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:28:04.762716+0000","last_peered":"2023-07-14T19:28:04.762716+0000","last_clean":"2023-07-14T19:28:04.762716+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:28:04.762716+0000","last_undegraded":"2023-07-14T19:28:04.762716+0000","last_fullsized":"2023-07-14T19:28:04.762716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:45.291302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758438+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:28:04.758438+0000","last_peered":"2023-07-14T19:28:04.758438+0000","last_clean":"2023-07-14T19:28:04.758438+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:28:04.758438+0000","last_undegraded":"2023-07-14T19:28:04.758438+0000","last_fullsized":"2023-07-14T19:28:04.758438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:34:18.122288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789986+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:27:05.789986+0000","last_peered":"2023-07-14T19:27:05.789986+0000","last_clean":"2023-07-14T19:27:05.789986+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:27:05.789986+0000","last_undegraded":"2023-07-14T19:27:05.789986+0000","last_fullsized":"2023-07-14T19:27:05.789986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.693215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762586+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:28:04.762586+0000","last_peered":"2023-07-14T19:28:04.762586+0000","last_clean":"2023-07-14T19:28:04.762586+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:28:04.762586+0000","last_undegraded":"2023-07-14T19:28:04.762586+0000","last_fullsized":"2023-07-14T19:28:04.762586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:10.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788456+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:27:05.788456+0000","last_peered":"2023-07-14T19:27:05.788456+0000","last_clean":"2023-07-14T19:27:05.788456+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:27:05.788456+0000","last_undegraded":"2023-07-14T19:27:05.788456+0000","last_fullsized":"2023-07-14T19:27:05.788456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:33.601716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768924+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:28:04.768924+0000","last_peered":"2023-07-14T19:28:04.768924+0000","last_clean":"2023-07-14T19:28:04.768924+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:28:04.768924+0000","last_undegraded":"2023-07-14T19:28:04.768924+0000","last_fullsized":"2023-07-14T19:28:04.768924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:47:04.807902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795938+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:05.795938+0000","last_peered":"2023-07-14T19:27:05.795938+0000","last_clean":"2023-07-14T19:27:05.795938+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:05.795938+0000","last_undegraded":"2023-07-14T19:27:05.795938+0000","last_fullsized":"2023-07-14T19:27:05.795938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:11.673840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.767910+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:28:04.767910+0000","last_peered":"2023-07-14T19:28:04.767910+0000","last_clean":"2023-07-14T19:28:04.767910+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:28:04.767910+0000","last_undegraded":"2023-07-14T19:28:04.767910+0000","last_fullsized":"2023-07-14T19:28:04.767910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:56.838822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795738+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:27:05.795738+0000","last_peered":"2023-07-14T19:27:05.795738+0000","last_clean":"2023-07-14T19:27:05.795738+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:27:05.795738+0000","last_undegraded":"2023-07-14T19:27:05.795738+0000","last_fullsized":"2023-07-14T19:27:05.795738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:12:12.590934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766461+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:28:04.766461+0000","last_peered":"2023-07-14T19:28:04.766461+0000","last_clean":"2023-07-14T19:28:04.766461+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:28:04.766461+0000","last_undegraded":"2023-07-14T19:28:04.766461+0000","last_fullsized":"2023-07-14T19:28:04.766461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:16.730207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.652776+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:28:19.652776+0000","last_peered":"2023-07-14T19:28:19.652776+0000","last_clean":"2023-07-14T19:28:19.652776+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:28:19.652776+0000","last_undegraded":"2023-07-14T19:28:19.652776+0000","last_fullsized":"2023-07-14T19:28:19.652776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:07.160367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759897+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:28:04.759897+0000","last_peered":"2023-07-14T19:28:04.759897+0000","last_clean":"2023-07-14T19:28:04.759897+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:28:04.759897+0000","last_undegraded":"2023-07-14T19:28:04.759897+0000","last_fullsized":"2023-07-14T19:28:04.759897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:02.082831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794601+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:27:05.794601+0000","last_peered":"2023-07-14T19:27:05.794601+0000","last_clean":"2023-07-14T19:27:05.794601+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:27:05.794601+0000","last_undegraded":"2023-07-14T19:27:05.794601+0000","last_fullsized":"2023-07-14T19:27:05.794601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:58.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760570+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:28:04.760570+0000","last_peered":"2023-07-14T19:28:04.760570+0000","last_clean":"2023-07-14T19:28:04.760570+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:28:04.760570+0000","last_undegraded":"2023-07-14T19:28:04.760570+0000","last_fullsized":"2023-07-14T19:28:04.760570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:02.114404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951680+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:05.951680+0000","last_peered":"2023-07-14T19:27:05.951680+0000","last_clean":"2023-07-14T19:27:05.951680+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:05.951680+0000","last_undegraded":"2023-07-14T19:27:05.951680+0000","last_fullsized":"2023-07-14T19:27:05.951680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.018579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758010+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:28:04.758010+0000","last_peered":"2023-07-14T19:28:04.758010+0000","last_clean":"2023-07-14T19:28:04.758010+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:28:04.758010+0000","last_undegraded":"2023-07-14T19:28:04.758010+0000","last_fullsized":"2023-07-14T19:28:04.758010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:26.867144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951987+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:27:05.951987+0000","last_peered":"2023-07-14T19:27:05.951987+0000","last_clean":"2023-07-14T19:27:05.951987+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:27:05.951987+0000","last_undegraded":"2023-07-14T19:27:05.951987+0000","last_fullsized":"2023-07-14T19:27:05.951987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.740777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758943+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:28:04.758943+0000","last_peered":"2023-07-14T19:28:04.758943+0000","last_clean":"2023-07-14T19:28:04.758943+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:28:04.758943+0000","last_undegraded":"2023-07-14T19:28:04.758943+0000","last_fullsized":"2023-07-14T19:28:04.758943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:42.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.652227+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:28:19.652227+0000","last_peered":"2023-07-14T19:28:19.652227+0000","last_clean":"2023-07-14T19:28:19.652227+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:28:19.652227+0000","last_undegraded":"2023-07-14T19:28:19.652227+0000","last_fullsized":"2023-07-14T19:28:19.652227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:38.911754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760323+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:28:04.760323+0000","last_peered":"2023-07-14T19:28:04.760323+0000","last_clean":"2023-07-14T19:28:04.760323+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:28:04.760323+0000","last_undegraded":"2023-07-14T19:28:04.760323+0000","last_fullsized":"2023-07-14T19:28:04.760323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:31.761093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952474+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:05.952474+0000","last_peered":"2023-07-14T19:27:05.952474+0000","last_clean":"2023-07-14T19:27:05.952474+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:05.952474+0000","last_undegraded":"2023-07-14T19:27:05.952474+0000","last_fullsized":"2023-07-14T19:27:05.952474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:41:09.562934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768566+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:28:04.768566+0000","last_peered":"2023-07-14T19:28:04.768566+0000","last_clean":"2023-07-14T19:28:04.768566+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:28:04.768566+0000","last_undegraded":"2023-07-14T19:28:04.768566+0000","last_fullsized":"2023-07-14T19:28:04.768566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:37:51.070907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952595+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:27:05.952595+0000","last_peered":"2023-07-14T19:27:05.952595+0000","last_clean":"2023-07-14T19:27:05.952595+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:27:05.952595+0000","last_undegraded":"2023-07-14T19:27:05.952595+0000","last_fullsized":"2023-07-14T19:27:05.952595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:10.381760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763772+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:28:04.763772+0000","last_peered":"2023-07-14T19:28:04.763772+0000","last_clean":"2023-07-14T19:28:04.763772+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:28:04.763772+0000","last_undegraded":"2023-07-14T19:28:04.763772+0000","last_fullsized":"2023-07-14T19:28:04.763772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:10.272127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":214,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:20.654387+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:28:20.654387+0000","last_peered":"2023-07-14T19:28:20.654387+0000","last_clean":"2023-07-14T19:28:20.654387+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:28:20.654387+0000","last_undegraded":"2023-07-14T19:28:20.654387+0000","last_fullsized":"2023-07-14T19:28:20.654387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:43.292916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56889,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":138,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756776+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:28:04.756776+0000","last_peered":"2023-07-14T19:28:04.756776+0000","last_clean":"2023-07-14T19:28:04.756776+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:28:04.756776+0000","last_undegraded":"2023-07-14T19:28:04.756776+0000","last_fullsized":"2023-07-14T19:28:04.756776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:55.206353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.804102+0000","last_change":"2023-07-14T19:27:05.803684+0000","last_active":"2023-07-14T19:27:05.804102+0000","last_peered":"2023-07-14T19:27:05.804102+0000","last_clean":"2023-07-14T19:27:05.804102+0000","last_became_active":"2023-07-14T19:27:05.796450+0000","last_became_peered":"2023-07-14T19:27:05.796450+0000","last_unstale":"2023-07-14T19:27:05.804102+0000","last_undegraded":"2023-07-14T19:27:05.804102+0000","last_fullsized":"2023-07-14T19:27:05.804102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:01.367033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.755135+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:28:04.755135+0000","last_peered":"2023-07-14T19:28:04.755135+0000","last_clean":"2023-07-14T19:28:04.755135+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:28:04.755135+0000","last_undegraded":"2023-07-14T19:28:04.755135+0000","last_fullsized":"2023-07-14T19:28:04.755135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:45:05.169932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952203+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:05.952203+0000","last_peered":"2023-07-14T19:27:05.952203+0000","last_clean":"2023-07-14T19:27:05.952203+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:05.952203+0000","last_undegraded":"2023-07-14T19:27:05.952203+0000","last_fullsized":"2023-07-14T19:27:05.952203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:08:51.150447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760213+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:28:04.760213+0000","last_peered":"2023-07-14T19:28:04.760213+0000","last_clean":"2023-07-14T19:28:04.760213+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:28:04.760213+0000","last_undegraded":"2023-07-14T19:28:04.760213+0000","last_fullsized":"2023-07-14T19:28:04.760213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:01:15.875961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":104,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.654308+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:28:19.654308+0000","last_peered":"2023-07-14T19:28:19.654308+0000","last_clean":"2023-07-14T19:28:19.654308+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:28:19.654308+0000","last_undegraded":"2023-07-14T19:28:19.654308+0000","last_fullsized":"2023-07-14T19:28:19.654308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:27.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764716+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:28:04.764716+0000","last_peered":"2023-07-14T19:28:04.764716+0000","last_clean":"2023-07-14T19:28:04.764716+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:28:04.764716+0000","last_undegraded":"2023-07-14T19:28:04.764716+0000","last_fullsized":"2023-07-14T19:28:04.764716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:10.265120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795312+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:05.795312+0000","last_peered":"2023-07-14T19:27:05.795312+0000","last_clean":"2023-07-14T19:27:05.795312+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:05.795312+0000","last_undegraded":"2023-07-14T19:27:05.795312+0000","last_fullsized":"2023-07-14T19:27:05.795312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:17:12.935696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763373+0000","last_change":"2023-07-14T19:27:05.906040+0000","last_active":"2023-07-14T19:28:04.763373+0000","last_peered":"2023-07-14T19:28:04.763373+0000","last_clean":"2023-07-14T19:28:04.763373+0000","last_became_active":"2023-07-14T19:27:05.905661+0000","last_became_peered":"2023-07-14T19:27:05.905661+0000","last_unstale":"2023-07-14T19:28:04.763373+0000","last_undegraded":"2023-07-14T19:28:04.763373+0000","last_fullsized":"2023-07-14T19:28:04.763373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:29.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795885+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:27:05.795885+0000","last_peered":"2023-07-14T19:27:05.795885+0000","last_clean":"2023-07-14T19:27:05.795885+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:27:05.795885+0000","last_undegraded":"2023-07-14T19:27:05.795885+0000","last_fullsized":"2023-07-14T19:27:05.795885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:46.077263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758100+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:28:04.758100+0000","last_peered":"2023-07-14T19:28:04.758100+0000","last_clean":"2023-07-14T19:28:04.758100+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:28:04.758100+0000","last_undegraded":"2023-07-14T19:28:04.758100+0000","last_fullsized":"2023-07-14T19:28:04.758100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:30.860044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.790247+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:27:05.790247+0000","last_peered":"2023-07-14T19:27:05.790247+0000","last_clean":"2023-07-14T19:27:05.790247+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:27:05.790247+0000","last_undegraded":"2023-07-14T19:27:05.790247+0000","last_fullsized":"2023-07-14T19:27:05.790247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:44:21.663771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758694+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:28:04.758694+0000","last_peered":"2023-07-14T19:28:04.758694+0000","last_clean":"2023-07-14T19:28:04.758694+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:28:04.758694+0000","last_undegraded":"2023-07-14T19:28:04.758694+0000","last_fullsized":"2023-07-14T19:28:04.758694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:38:26.970479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952264+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:27:05.952264+0000","last_peered":"2023-07-14T19:27:05.952264+0000","last_clean":"2023-07-14T19:27:05.952264+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:27:05.952264+0000","last_undegraded":"2023-07-14T19:27:05.952264+0000","last_fullsized":"2023-07-14T19:27:05.952264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:28.081451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769444+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:28:04.769444+0000","last_peered":"2023-07-14T19:28:04.769444+0000","last_clean":"2023-07-14T19:28:04.769444+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:28:04.769444+0000","last_undegraded":"2023-07-14T19:28:04.769444+0000","last_fullsized":"2023-07-14T19:28:04.769444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:40.592640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.752609+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:28:04.752609+0000","last_peered":"2023-07-14T19:28:04.752609+0000","last_clean":"2023-07-14T19:28:04.752609+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:28:04.752609+0000","last_undegraded":"2023-07-14T19:28:04.752609+0000","last_fullsized":"2023-07-14T19:28:04.752609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:29.089446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:20.654935+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:28:20.654935+0000","last_peered":"2023-07-14T19:28:20.654935+0000","last_clean":"2023-07-14T19:28:20.654935+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:28:20.654935+0000","last_undegraded":"2023-07-14T19:28:20.654935+0000","last_fullsized":"2023-07-14T19:28:20.654935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:52:38.820059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765118+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:28:04.765118+0000","last_peered":"2023-07-14T19:28:04.765118+0000","last_clean":"2023-07-14T19:28:04.765118+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:28:04.765118+0000","last_undegraded":"2023-07-14T19:28:04.765118+0000","last_fullsized":"2023-07-14T19:28:04.765118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:20.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789079+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:27:05.789079+0000","last_peered":"2023-07-14T19:27:05.789079+0000","last_clean":"2023-07-14T19:27:05.789079+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:27:05.789079+0000","last_undegraded":"2023-07-14T19:27:05.789079+0000","last_fullsized":"2023-07-14T19:27:05.789079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:11.895409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952086+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:27:05.952086+0000","last_peered":"2023-07-14T19:27:05.952086+0000","last_clean":"2023-07-14T19:27:05.952086+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:27:05.952086+0000","last_undegraded":"2023-07-14T19:27:05.952086+0000","last_fullsized":"2023-07-14T19:27:05.952086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:57.503183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756998+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:28:04.756998+0000","last_peered":"2023-07-14T19:28:04.756998+0000","last_clean":"2023-07-14T19:28:04.756998+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:28:04.756998+0000","last_undegraded":"2023-07-14T19:28:04.756998+0000","last_fullsized":"2023-07-14T19:28:04.756998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:37.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952256+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:05.952256+0000","last_peered":"2023-07-14T19:27:05.952256+0000","last_clean":"2023-07-14T19:27:05.952256+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:05.952256+0000","last_undegraded":"2023-07-14T19:27:05.952256+0000","last_fullsized":"2023-07-14T19:27:05.952256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.155552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795661+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:05.795661+0000","last_peered":"2023-07-14T19:27:05.795661+0000","last_clean":"2023-07-14T19:27:05.795661+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:05.795661+0000","last_undegraded":"2023-07-14T19:27:05.795661+0000","last_fullsized":"2023-07-14T19:27:05.795661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:05.271093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760036+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:28:04.760036+0000","last_peered":"2023-07-14T19:28:04.760036+0000","last_clean":"2023-07-14T19:28:04.760036+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:28:04.760036+0000","last_undegraded":"2023-07-14T19:28:04.760036+0000","last_fullsized":"2023-07-14T19:28:04.760036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:12:18.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:10.903427+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:27:10.903427+0000","last_peered":"2023-07-14T19:27:10.903427+0000","last_clean":"2023-07-14T19:27:10.903427+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:27:10.903427+0000","last_undegraded":"2023-07-14T19:27:10.903427+0000","last_fullsized":"2023-07-14T19:27:10.903427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:26.769682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762774+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:28:04.762774+0000","last_peered":"2023-07-14T19:28:04.762774+0000","last_clean":"2023-07-14T19:28:04.762774+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:28:04.762774+0000","last_undegraded":"2023-07-14T19:28:04.762774+0000","last_fullsized":"2023-07-14T19:28:04.762774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:14.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788996+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:27:05.788996+0000","last_peered":"2023-07-14T19:27:05.788996+0000","last_clean":"2023-07-14T19:27:05.788996+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:27:05.788996+0000","last_undegraded":"2023-07-14T19:27:05.788996+0000","last_fullsized":"2023-07-14T19:27:05.788996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:04:44.601003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.767489+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:28:04.767489+0000","last_peered":"2023-07-14T19:28:04.767489+0000","last_clean":"2023-07-14T19:28:04.767489+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:28:04.767489+0000","last_undegraded":"2023-07-14T19:28:04.767489+0000","last_fullsized":"2023-07-14T19:28:04.767489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:45:38.640268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903756+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:05.903756+0000","last_peered":"2023-07-14T19:27:05.903756+0000","last_clean":"2023-07-14T19:27:05.903756+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:05.903756+0000","last_undegraded":"2023-07-14T19:27:05.903756+0000","last_fullsized":"2023-07-14T19:27:05.903756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:31.831826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.739189+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:28:04.739189+0000","last_peered":"2023-07-14T19:28:04.739189+0000","last_clean":"2023-07-14T19:28:04.739189+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:28:04.739189+0000","last_undegraded":"2023-07-14T19:28:04.739189+0000","last_fullsized":"2023-07-14T19:28:04.739189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:50.824483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787960+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:05.787960+0000","last_peered":"2023-07-14T19:27:05.787960+0000","last_clean":"2023-07-14T19:27:05.787960+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:05.787960+0000","last_undegraded":"2023-07-14T19:27:05.787960+0000","last_fullsized":"2023-07-14T19:27:05.787960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:47.286623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764282+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:28:04.764282+0000","last_peered":"2023-07-14T19:28:04.764282+0000","last_clean":"2023-07-14T19:28:04.764282+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:28:04.764282+0000","last_undegraded":"2023-07-14T19:28:04.764282+0000","last_fullsized":"2023-07-14T19:28:04.764282+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:20.246759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.806328+0000","last_change":"2023-07-14T19:27:05.805846+0000","last_active":"2023-07-14T19:27:05.806328+0000","last_peered":"2023-07-14T19:27:05.806328+0000","last_clean":"2023-07-14T19:27:05.806328+0000","last_became_active":"2023-07-14T19:27:05.796805+0000","last_became_peered":"2023-07-14T19:27:05.796805+0000","last_unstale":"2023-07-14T19:27:05.806328+0000","last_undegraded":"2023-07-14T19:27:05.806328+0000","last_fullsized":"2023-07-14T19:27:05.806328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:47:58.264439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.771308+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:28:04.771308+0000","last_peered":"2023-07-14T19:28:04.771308+0000","last_clean":"2023-07-14T19:28:04.771308+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:28:04.771308+0000","last_undegraded":"2023-07-14T19:28:04.771308+0000","last_fullsized":"2023-07-14T19:28:04.771308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:54:50.760237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795490+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:05.795490+0000","last_peered":"2023-07-14T19:27:05.795490+0000","last_clean":"2023-07-14T19:27:05.795490+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:05.795490+0000","last_undegraded":"2023-07-14T19:27:05.795490+0000","last_fullsized":"2023-07-14T19:27:05.795490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:10:28.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760928+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:28:04.760928+0000","last_peered":"2023-07-14T19:28:04.760928+0000","last_clean":"2023-07-14T19:28:04.760928+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:28:04.760928+0000","last_undegraded":"2023-07-14T19:28:04.760928+0000","last_fullsized":"2023-07-14T19:28:04.760928+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:47:05.779967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795623+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:05.795623+0000","last_peered":"2023-07-14T19:27:05.795623+0000","last_clean":"2023-07-14T19:27:05.795623+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:05.795623+0000","last_undegraded":"2023-07-14T19:27:05.795623+0000","last_fullsized":"2023-07-14T19:27:05.795623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:42.189785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766892+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:28:04.766892+0000","last_peered":"2023-07-14T19:28:04.766892+0000","last_clean":"2023-07-14T19:28:04.766892+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:28:04.766892+0000","last_undegraded":"2023-07-14T19:28:04.766892+0000","last_fullsized":"2023-07-14T19:28:04.766892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:31:52.615160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952458+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:27:05.952458+0000","last_peered":"2023-07-14T19:27:05.952458+0000","last_clean":"2023-07-14T19:27:05.952458+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:27:05.952458+0000","last_undegraded":"2023-07-14T19:27:05.952458+0000","last_fullsized":"2023-07-14T19:27:05.952458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:34.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764119+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:28:04.764119+0000","last_peered":"2023-07-14T19:28:04.764119+0000","last_clean":"2023-07-14T19:28:04.764119+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:28:04.764119+0000","last_undegraded":"2023-07-14T19:28:04.764119+0000","last_fullsized":"2023-07-14T19:28:04.764119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:50:19.375696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951776+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:27:05.951776+0000","last_peered":"2023-07-14T19:27:05.951776+0000","last_clean":"2023-07-14T19:27:05.951776+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:27:05.951776+0000","last_undegraded":"2023-07-14T19:27:05.951776+0000","last_fullsized":"2023-07-14T19:27:05.951776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:55.101835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758103+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:28:04.758103+0000","last_peered":"2023-07-14T19:28:04.758103+0000","last_clean":"2023-07-14T19:28:04.758103+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:28:04.758103+0000","last_undegraded":"2023-07-14T19:28:04.758103+0000","last_fullsized":"2023-07-14T19:28:04.758103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:56:02.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762361+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:28:04.762361+0000","last_peered":"2023-07-14T19:28:04.762361+0000","last_clean":"2023-07-14T19:28:04.762361+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:28:04.762361+0000","last_undegraded":"2023-07-14T19:28:04.762361+0000","last_fullsized":"2023-07-14T19:28:04.762361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:03:59.337396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759168+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:28:04.759168+0000","last_peered":"2023-07-14T19:28:04.759168+0000","last_clean":"2023-07-14T19:28:04.759168+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:28:04.759168+0000","last_undegraded":"2023-07-14T19:28:04.759168+0000","last_fullsized":"2023-07-14T19:28:04.759168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:26:45.117785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763076+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:28:04.763076+0000","last_peered":"2023-07-14T19:28:04.763076+0000","last_clean":"2023-07-14T19:28:04.763076+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:28:04.763076+0000","last_undegraded":"2023-07-14T19:28:04.763076+0000","last_fullsized":"2023-07-14T19:28:04.763076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:59.431251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.773029+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:28:04.773029+0000","last_peered":"2023-07-14T19:28:04.773029+0000","last_clean":"2023-07-14T19:28:04.773029+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:28:04.773029+0000","last_undegraded":"2023-07-14T19:28:04.773029+0000","last_fullsized":"2023-07-14T19:28:04.773029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.135050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764144+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:28:04.764144+0000","last_peered":"2023-07-14T19:28:04.764144+0000","last_clean":"2023-07-14T19:28:04.764144+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:28:04.764144+0000","last_undegraded":"2023-07-14T19:28:04.764144+0000","last_fullsized":"2023-07-14T19:28:04.764144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:24.075259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760086+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:28:04.760086+0000","last_peered":"2023-07-14T19:28:04.760086+0000","last_clean":"2023-07-14T19:28:04.760086+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:28:04.760086+0000","last_undegraded":"2023-07-14T19:28:04.760086+0000","last_fullsized":"2023-07-14T19:28:04.760086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:50.553918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.750630+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:28:04.750630+0000","last_peered":"2023-07-14T19:28:04.750630+0000","last_clean":"2023-07-14T19:28:04.750630+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:28:04.750630+0000","last_undegraded":"2023-07-14T19:28:04.750630+0000","last_fullsized":"2023-07-14T19:28:04.750630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:43:45.456973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756375+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:28:04.756375+0000","last_peered":"2023-07-14T19:28:04.756375+0000","last_clean":"2023-07-14T19:28:04.756375+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:28:04.756375+0000","last_undegraded":"2023-07-14T19:28:04.756375+0000","last_fullsized":"2023-07-14T19:28:04.756375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:43.308371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.761507+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:28:04.761507+0000","last_peered":"2023-07-14T19:28:04.761507+0000","last_clean":"2023-07-14T19:28:04.761507+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:28:04.761507+0000","last_undegraded":"2023-07-14T19:28:04.761507+0000","last_fullsized":"2023-07-14T19:28:04.761507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:02:45.526660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759622+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:28:04.759622+0000","last_peered":"2023-07-14T19:28:04.759622+0000","last_clean":"2023-07-14T19:28:04.759622+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:28:04.759622+0000","last_undegraded":"2023-07-14T19:28:04.759622+0000","last_fullsized":"2023-07-14T19:28:04.759622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:50.377751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765248+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:28:04.765248+0000","last_peered":"2023-07-14T19:28:04.765248+0000","last_clean":"2023-07-14T19:28:04.765248+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:28:04.765248+0000","last_undegraded":"2023-07-14T19:28:04.765248+0000","last_fullsized":"2023-07-14T19:28:04.765248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:51:09.749154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769059+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:28:04.769059+0000","last_peered":"2023-07-14T19:28:04.769059+0000","last_clean":"2023-07-14T19:28:04.769059+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:28:04.769059+0000","last_undegraded":"2023-07-14T19:28:04.769059+0000","last_fullsized":"2023-07-14T19:28:04.769059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:02.143813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759931+0000","last_change":"2023-07-14T19:27:05.796530+0000","last_active":"2023-07-14T19:28:04.759931+0000","last_peered":"2023-07-14T19:28:04.759931+0000","last_clean":"2023-07-14T19:28:04.759931+0000","last_became_active":"2023-07-14T19:27:05.796148+0000","last_became_peered":"2023-07-14T19:27:05.796148+0000","last_unstale":"2023-07-14T19:28:04.759931+0000","last_undegraded":"2023-07-14T19:28:04.759931+0000","last_fullsized":"2023-07-14T19:28:04.759931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:52.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61252,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":505,"num_read_kb":318,"num_write":157,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":192078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514525,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69499999999999995,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.499},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.39000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":5.8380000000000001,"15min":5.8380000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.79300000000000004,"5min":0.874,"15min":0.874},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Jul 14 19:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.53000000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.56499999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.87,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.88700000000000001,"5min":1.179,"15min":1.179},"last":0.86699999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.90500000000000003}]},{"osd":9,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89900000000000002,"15min":0.91300000000000003},"min":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96599999999999997,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.98199999999999998,"15min":1.0149999999999999},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.89400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.998,"5min":1.083,"15min":1.083},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.82799999999999996,"15min":0.82299999999999995},"min":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.016,"5min":1.375,"15min":1.375},"last":0.88}]}]},{"osd":10,"up_from":69,"seq":296352743457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1080420,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.47099999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.69799999999999995,"15min":0.72299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":0.998,"15min":0.998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.66400000000000003,"15min":0.68400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70299999999999996,"5min":1.222,"15min":1.222},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67000000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78200000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.879,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.5,"15min":0.5},"max":{"1min":0.89700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82399999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83599999999999997,"5min":1.24,"15min":1.24},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79700000000000004,"15min":0.80500000000000005},"min":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88200000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.874},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95799999999999996,"5min":1.095,"15min":1.095},"last":0.72899999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.82399999999999995,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.88100000000000001},"min":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93000000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Jul 14 19:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.93100000000000005,"5min":1.02,"15min":1.02},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.627,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.94599999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.89200000000000002}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":432010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57499999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55600000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.438}]},{"osd":1,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79000000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77900000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.70699999999999996,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72199999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67700000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67100000000000004,"5min":0.753,"15min":0.753},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76300000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89200000000000002,"5min":1.081,"15min":1.081},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":1.087,"15min":1.087},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.74299999999999999,"15min":0.76200000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81999999999999995,"5min":1.141,"15min":1.141},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83499999999999996,"5min":1.367,"15min":1.367},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 14 19:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.90300000000000002,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.86599999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.70399999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":431930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64000000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.5,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.504}]},{"osd":3,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.751,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88700000000000001,"5min":1.534,"15min":1.534},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.628},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86599999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95599999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.675,"15min":1.675},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.93500000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.56699999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.627,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.75,"15min":0.72599999999999998},"min":{"1min":0.72499999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0189999999999999,"5min":1.03,"15min":1.03},"last":0.99299999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83299999999999996}]}]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1023579,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.5,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.502,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84799999999999998,"5min":1.024,"15min":1.024},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.47399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82599999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92500000000000004,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.52600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97499999999999998,"5min":1.127,"15min":1.127},"last":0.86399999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.91400000000000003,"5min":1.954,"15min":1.954},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.73699999999999999,"15min":0.70599999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.878,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93000000000000005,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":1,"15min":1},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.76900000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.7610000000000001,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.93899999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94599999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91100000000000003}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.98899999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":431405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.374},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.54200000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":1.212,"15min":1.212},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.63800000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56100000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75600000000000001,"5min":1.115,"15min":1.115},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.032,"15min":1.032},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70399999999999996,"15min":0.72199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.077,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93100000000000005,"5min":1.151,"15min":1.151},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.68500000000000005,"15min":0.72199999999999998},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.68200000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.68799999999999994,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.59999999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69699999999999995,"15min":0.71999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3480000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67600000000000005,"5min":1.55,"15min":1.55},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.746,"15min":0.76400000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":492401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.36199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57799999999999996,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.876},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57199999999999995,"15min":0.51200000000000001},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84699999999999998,"5min":1.302,"15min":1.302},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.41899999999999998},"min":{"1min":0.41199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.626},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.48999999999999999,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63100000000000001,"15min":0.58499999999999996},"min":{"1min":0.51200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.627,"15min":0.57899999999999996},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71899999999999997,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.60699999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.879,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.58699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.98499999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.69199999999999995,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67900000000000005,"15min":0.63900000000000001},"min":{"1min":0.60599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.66000000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.93600000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":1.087,"15min":1.087},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.65700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.996}]},{"osd":11,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":1.042},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.0509999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":432670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.48699999999999999},"min":{"1min":0.34999999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.874,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.749,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.55400000000000005},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.376},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.435}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71199999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76900000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77300000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65600000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.77600000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":494200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.47599999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.55200000000000005,"15min":0.48699999999999999},"min":{"1min":0.40799999999999997,"5min":0.316,"15min":0.316},"max":{"1min":3.8759999999999999,"5min":3.8759999999999999,"15min":3.8759999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57499999999999996,"15min":0.51800000000000002},"min":{"1min":0.40699999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58699999999999997,"15min":0.52400000000000002},"min":{"1min":0.375,"5min":0.255,"15min":0.255},"max":{"1min":0.83899999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60799999999999998,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.436},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.9359999999999999,"5min":4.9359999999999999,"15min":4.9359999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.45100000000000001},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.56100000000000005,"15min":0.52500000000000002},"min":{"1min":0.52400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.7560000000000002,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.753},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.69699999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.65400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.754,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149757,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":431722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32100000000000001},"min":{"1min":0.28299999999999997,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55300000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68000000000000005,"5min":1.202,"15min":1.202},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.503},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.46100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.54300000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.442},{"interface":"front","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.373}]},{"osd":7,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78800000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.71199999999999997,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.006,"15min":1.006},"last":0.66500000000000004}]},{"osd":11,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67500000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.39300000000000002,"15min":0.372},"min":{"1min":0.32200000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.81000000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.52800000000000002},"min":{"1min":0.44700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.41399999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58199999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85899999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61099999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1023311,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.68200000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.996,"15min":0.996},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.42799999999999999},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":1.022,"15min":1.022},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77100000000000002,"5min":1.044,"15min":1.044},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64700000000000002,"15min":0.628},"min":{"1min":0.58799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.624,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91700000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81299999999999994,"5min":1.087,"15min":1.087},"last":0.749},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":1.113,"15min":1.113},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Jul 14 19:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95299999999999996,"5min":1.004,"15min":1.004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.59399999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.91400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.46700000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93799999999999994,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.57899999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.875}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Jul 14 19:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.73699999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.753,"15min":0.747},"min":{"1min":0.64600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:28:23.394 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T19:28:23.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 -- 172.21.15.82:0/3079551064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 msgr2=0x7fce3c107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 --2- 172.21.15.82:0/3079551064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c107d40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fce3400b0a0 tx=0x7fce3400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 -- 172.21.15.82:0/3079551064 shutdown_connections 2023-07-14T19:28:23.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 --2- 172.21.15.82:0/3079551064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c107d40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 --2- 172.21.15.82:0/3079551064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce3c1006a0 0x7fce3c102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 --2- 172.21.15.82:0/3079551064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c102fc0 0x7fce3c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 -- 172.21.15.82:0/3079551064 >> 172.21.15.82:0/3079551064 conn(0x7fce3c0fa100 msgr2=0x7fce3c0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:23.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 -- 172.21.15.82:0/3079551064 shutdown_connections 2023-07-14T19:28:23.555 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.547+0000 7fce41367700 1 -- 172.21.15.82:0/3079551064 wait complete. 2023-07-14T19:28:23.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 Processor -- start 2023-07-14T19:28:23.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 -- start start 2023-07-14T19:28:23.556 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c1006a0 0x7fce3c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce3c102fc0 0x7fce3c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19f710 con 0x7fce3c1058e0 2023-07-14T19:28:23.557 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19f850 con 0x7fce3c102fc0 2023-07-14T19:28:23.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce41367700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19f9e0 con 0x7fce3c1006a0 2023-07-14T19:28:23.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.558 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42190/0 (socket says 172.21.15.82:42190) 2023-07-14T19:28:23.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 -- 172.21.15.82:0/4047866694 learned_addr learned my addr 172.21.15.82:0/4047866694 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:23.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3a7fc700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce3c102fc0 0x7fce3c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3affd700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c1006a0 0x7fce3c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 -- 172.21.15.82:0/4047866694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c1006a0 msgr2=0x7fce3c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c1006a0 0x7fce3c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 -- 172.21.15.82:0/4047866694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce3c102fc0 msgr2=0x7fce3c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce3c102fc0 0x7fce3c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.560 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3b7fe700 1 -- 172.21.15.82:0/4047866694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009d00 con 0x7fce3c1058e0 2023-07-14T19:28:23.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.548+0000 7fce3affd700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c1006a0 0x7fce3c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:23.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.549+0000 7fce3b7fe700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c19f140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce3400b780 tx=0x7fce3400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:23.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.549+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400e040 con 0x7fce3c1058e0 2023-07-14T19:28:23.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.549+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce34004c50 con 0x7fce3c1058e0 2023-07-14T19:28:23.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.549+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c19fc60 con 0x7fce3c1058e0 2023-07-14T19:28:23.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.549+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340053b0 con 0x7fce3c1058e0 2023-07-14T19:28:23.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.549+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c1a0130 con 0x7fce3c1058e0 2023-07-14T19:28:23.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.550+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c107450 con 0x7fce3c1058e0 2023-07-14T19:28:23.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.551+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fce3401d9c0 con 0x7fce3c1058e0 2023-07-14T19:28:23.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.551+0000 7fce23fff700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce240a1220 0x7fce240a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.551+0000 7fce3affd700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce240a1220 0x7fce240a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.551+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fce340cdd50 con 0x7fce3c1058e0 2023-07-14T19:28:23.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.552+0000 7fce3affd700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce240a1220 0x7fce240a36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009cb0 tx=0x7fce2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:23.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.555+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fce340d1030 con 0x7fce3c1058e0 2023-07-14T19:28:23.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.743+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce3c060a80 con 0x7fce240a1220 2023-07-14T19:28:23.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.753+0000 7fce23fff700 1 -- 172.21.15.82:0/4047866694 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344000 (secure 0 0 0) 0x7fce3c060a80 con 0x7fce240a1220 2023-07-14T19:28:23.757 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:23.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce240a1220 msgr2=0x7fce240a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce240a1220 0x7fce240a36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009cb0 tx=0x7fce2c009260 comp rx=0 tx=0).stop 2023-07-14T19:28:23.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 msgr2=0x7fce3c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c19f140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce3400b780 tx=0x7fce3400bab0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 shutdown_connections 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fce240a1220 0x7fce240a36d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce3c1058e0 0x7fce3c19f140 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce3c1006a0 0x7fce3c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 --2- 172.21.15.82:0/4047866694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce3c102fc0 0x7fce3c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 >> 172.21.15.82:0/4047866694 conn(0x7fce3c0fa100 msgr2=0x7fce3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:23.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 shutdown_connections 2023-07-14T19:28:23.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.756+0000 7fce41367700 1 -- 172.21.15.82:0/4047866694 wait complete. 2023-07-14T19:28:23.761 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:28:23.770 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-07-14T19:28:22.738141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651866,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":641,"num_read_kb":437,"num_write":340,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":777,"ondisk_log_size":777,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335172,"kb_used_data":10980,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606652,"statfs":{"total":1151940427776,"available":1151597211648,"internally_reserved":0,"allocated":11243520,"data_stored":7137562,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":47,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766460+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:28:04.766460+0000","last_peered":"2023-07-14T19:28:04.766460+0000","last_clean":"2023-07-14T19:28:04.766460+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:28:04.766460+0000","last_undegraded":"2023-07-14T19:28:04.766460+0000","last_fullsized":"2023-07-14T19:28:04.766460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:41.471339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769021+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:28:04.769021+0000","last_peered":"2023-07-14T19:28:04.769021+0000","last_clean":"2023-07-14T19:28:04.769021+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:28:04.769021+0000","last_undegraded":"2023-07-14T19:28:04.769021+0000","last_fullsized":"2023-07-14T19:28:04.769021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:01.099918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765502+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:28:04.765502+0000","last_peered":"2023-07-14T19:28:04.765502+0000","last_clean":"2023-07-14T19:28:04.765502+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:28:04.765502+0000","last_undegraded":"2023-07-14T19:28:04.765502+0000","last_fullsized":"2023-07-14T19:28:04.765502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:33:44.231618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.761968+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:28:04.761968+0000","last_peered":"2023-07-14T19:28:04.761968+0000","last_clean":"2023-07-14T19:28:04.761968+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:28:04.761968+0000","last_undegraded":"2023-07-14T19:28:04.761968+0000","last_fullsized":"2023-07-14T19:28:04.761968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:38:02.561344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.757252+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:28:04.757252+0000","last_peered":"2023-07-14T19:28:04.757252+0000","last_clean":"2023-07-14T19:28:04.757252+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:28:04.757252+0000","last_undegraded":"2023-07-14T19:28:04.757252+0000","last_fullsized":"2023-07-14T19:28:04.757252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:36:44.500967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759197+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:28:04.759197+0000","last_peered":"2023-07-14T19:28:04.759197+0000","last_clean":"2023-07-14T19:28:04.759197+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:28:04.759197+0000","last_undegraded":"2023-07-14T19:28:04.759197+0000","last_fullsized":"2023-07-14T19:28:04.759197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.697906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764042+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:28:04.764042+0000","last_peered":"2023-07-14T19:28:04.764042+0000","last_clean":"2023-07-14T19:28:04.764042+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:28:04.764042+0000","last_undegraded":"2023-07-14T19:28:04.764042+0000","last_fullsized":"2023-07-14T19:28:04.764042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768382+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:28:04.768382+0000","last_peered":"2023-07-14T19:28:04.768382+0000","last_clean":"2023-07-14T19:28:04.768382+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:28:04.768382+0000","last_undegraded":"2023-07-14T19:28:04.768382+0000","last_fullsized":"2023-07-14T19:28:04.768382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:11.270082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765502+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:28:04.765502+0000","last_peered":"2023-07-14T19:28:04.765502+0000","last_clean":"2023-07-14T19:28:04.765502+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:28:04.765502+0000","last_undegraded":"2023-07-14T19:28:04.765502+0000","last_fullsized":"2023-07-14T19:28:04.765502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:57:14.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758142+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:28:04.758142+0000","last_peered":"2023-07-14T19:28:04.758142+0000","last_clean":"2023-07-14T19:28:04.758142+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:28:04.758142+0000","last_undegraded":"2023-07-14T19:28:04.758142+0000","last_fullsized":"2023-07-14T19:28:04.758142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:09.380546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766790+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:28:04.766790+0000","last_peered":"2023-07-14T19:28:04.766790+0000","last_clean":"2023-07-14T19:28:04.766790+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:28:04.766790+0000","last_undegraded":"2023-07-14T19:28:04.766790+0000","last_fullsized":"2023-07-14T19:28:04.766790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:24.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763507+0000","last_change":"2023-07-14T19:27:05.797079+0000","last_active":"2023-07-14T19:28:04.763507+0000","last_peered":"2023-07-14T19:28:04.763507+0000","last_clean":"2023-07-14T19:28:04.763507+0000","last_became_active":"2023-07-14T19:27:05.796765+0000","last_became_peered":"2023-07-14T19:27:05.796765+0000","last_unstale":"2023-07-14T19:28:04.763507+0000","last_undegraded":"2023-07-14T19:28:04.763507+0000","last_fullsized":"2023-07-14T19:28:04.763507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:18.699215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760253+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:28:04.760253+0000","last_peered":"2023-07-14T19:28:04.760253+0000","last_clean":"2023-07-14T19:28:04.760253+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:28:04.760253+0000","last_undegraded":"2023-07-14T19:28:04.760253+0000","last_fullsized":"2023-07-14T19:28:04.760253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:24.688894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768895+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:28:04.768895+0000","last_peered":"2023-07-14T19:28:04.768895+0000","last_clean":"2023-07-14T19:28:04.768895+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:28:04.768895+0000","last_undegraded":"2023-07-14T19:28:04.768895+0000","last_fullsized":"2023-07-14T19:28:04.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:11:02.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768383+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:28:04.768383+0000","last_peered":"2023-07-14T19:28:04.768383+0000","last_clean":"2023-07-14T19:28:04.768383+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:28:04.768383+0000","last_undegraded":"2023-07-14T19:28:04.768383+0000","last_fullsized":"2023-07-14T19:28:04.768383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:29.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764585+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:28:04.764585+0000","last_peered":"2023-07-14T19:28:04.764585+0000","last_clean":"2023-07-14T19:28:04.764585+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:28:04.764585+0000","last_undegraded":"2023-07-14T19:28:04.764585+0000","last_fullsized":"2023-07-14T19:28:04.764585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:14.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766558+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:28:04.766558+0000","last_peered":"2023-07-14T19:28:04.766558+0000","last_clean":"2023-07-14T19:28:04.766558+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:28:04.766558+0000","last_undegraded":"2023-07-14T19:28:04.766558+0000","last_fullsized":"2023-07-14T19:28:04.766558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:47.623791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762716+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:28:04.762716+0000","last_peered":"2023-07-14T19:28:04.762716+0000","last_clean":"2023-07-14T19:28:04.762716+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:28:04.762716+0000","last_undegraded":"2023-07-14T19:28:04.762716+0000","last_fullsized":"2023-07-14T19:28:04.762716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:45.291302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758438+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:28:04.758438+0000","last_peered":"2023-07-14T19:28:04.758438+0000","last_clean":"2023-07-14T19:28:04.758438+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:28:04.758438+0000","last_undegraded":"2023-07-14T19:28:04.758438+0000","last_fullsized":"2023-07-14T19:28:04.758438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:34:18.122288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789986+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:27:05.789986+0000","last_peered":"2023-07-14T19:27:05.789986+0000","last_clean":"2023-07-14T19:27:05.789986+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:27:05.789986+0000","last_undegraded":"2023-07-14T19:27:05.789986+0000","last_fullsized":"2023-07-14T19:27:05.789986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.693215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762586+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:28:04.762586+0000","last_peered":"2023-07-14T19:28:04.762586+0000","last_clean":"2023-07-14T19:28:04.762586+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:28:04.762586+0000","last_undegraded":"2023-07-14T19:28:04.762586+0000","last_fullsized":"2023-07-14T19:28:04.762586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:10.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788456+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:27:05.788456+0000","last_peered":"2023-07-14T19:27:05.788456+0000","last_clean":"2023-07-14T19:27:05.788456+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:27:05.788456+0000","last_undegraded":"2023-07-14T19:27:05.788456+0000","last_fullsized":"2023-07-14T19:27:05.788456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:33.601716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768924+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:28:04.768924+0000","last_peered":"2023-07-14T19:28:04.768924+0000","last_clean":"2023-07-14T19:28:04.768924+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:28:04.768924+0000","last_undegraded":"2023-07-14T19:28:04.768924+0000","last_fullsized":"2023-07-14T19:28:04.768924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:47:04.807902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795938+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:05.795938+0000","last_peered":"2023-07-14T19:27:05.795938+0000","last_clean":"2023-07-14T19:27:05.795938+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:05.795938+0000","last_undegraded":"2023-07-14T19:27:05.795938+0000","last_fullsized":"2023-07-14T19:27:05.795938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:11.673840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.767910+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:28:04.767910+0000","last_peered":"2023-07-14T19:28:04.767910+0000","last_clean":"2023-07-14T19:28:04.767910+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:28:04.767910+0000","last_undegraded":"2023-07-14T19:28:04.767910+0000","last_fullsized":"2023-07-14T19:28:04.767910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:56.838822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795738+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:27:05.795738+0000","last_peered":"2023-07-14T19:27:05.795738+0000","last_clean":"2023-07-14T19:27:05.795738+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:27:05.795738+0000","last_undegraded":"2023-07-14T19:27:05.795738+0000","last_fullsized":"2023-07-14T19:27:05.795738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:12:12.590934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766461+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:28:04.766461+0000","last_peered":"2023-07-14T19:28:04.766461+0000","last_clean":"2023-07-14T19:28:04.766461+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:28:04.766461+0000","last_undegraded":"2023-07-14T19:28:04.766461+0000","last_fullsized":"2023-07-14T19:28:04.766461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:16.730207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.652776+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:28:19.652776+0000","last_peered":"2023-07-14T19:28:19.652776+0000","last_clean":"2023-07-14T19:28:19.652776+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:28:19.652776+0000","last_undegraded":"2023-07-14T19:28:19.652776+0000","last_fullsized":"2023-07-14T19:28:19.652776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:07.160367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759897+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:28:04.759897+0000","last_peered":"2023-07-14T19:28:04.759897+0000","last_clean":"2023-07-14T19:28:04.759897+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:28:04.759897+0000","last_undegraded":"2023-07-14T19:28:04.759897+0000","last_fullsized":"2023-07-14T19:28:04.759897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:02.082831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794601+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:27:05.794601+0000","last_peered":"2023-07-14T19:27:05.794601+0000","last_clean":"2023-07-14T19:27:05.794601+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:27:05.794601+0000","last_undegraded":"2023-07-14T19:27:05.794601+0000","last_fullsized":"2023-07-14T19:27:05.794601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:58.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760570+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:28:04.760570+0000","last_peered":"2023-07-14T19:28:04.760570+0000","last_clean":"2023-07-14T19:28:04.760570+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:28:04.760570+0000","last_undegraded":"2023-07-14T19:28:04.760570+0000","last_fullsized":"2023-07-14T19:28:04.760570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:02.114404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951680+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:05.951680+0000","last_peered":"2023-07-14T19:27:05.951680+0000","last_clean":"2023-07-14T19:27:05.951680+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:05.951680+0000","last_undegraded":"2023-07-14T19:27:05.951680+0000","last_fullsized":"2023-07-14T19:27:05.951680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.018579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758010+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:28:04.758010+0000","last_peered":"2023-07-14T19:28:04.758010+0000","last_clean":"2023-07-14T19:28:04.758010+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:28:04.758010+0000","last_undegraded":"2023-07-14T19:28:04.758010+0000","last_fullsized":"2023-07-14T19:28:04.758010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:26.867144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951987+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:27:05.951987+0000","last_peered":"2023-07-14T19:27:05.951987+0000","last_clean":"2023-07-14T19:27:05.951987+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:27:05.951987+0000","last_undegraded":"2023-07-14T19:27:05.951987+0000","last_fullsized":"2023-07-14T19:27:05.951987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.740777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758943+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:28:04.758943+0000","last_peered":"2023-07-14T19:28:04.758943+0000","last_clean":"2023-07-14T19:28:04.758943+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:28:04.758943+0000","last_undegraded":"2023-07-14T19:28:04.758943+0000","last_fullsized":"2023-07-14T19:28:04.758943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:42.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.652227+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:28:19.652227+0000","last_peered":"2023-07-14T19:28:19.652227+0000","last_clean":"2023-07-14T19:28:19.652227+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:28:19.652227+0000","last_undegraded":"2023-07-14T19:28:19.652227+0000","last_fullsized":"2023-07-14T19:28:19.652227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:38.911754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760323+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:28:04.760323+0000","last_peered":"2023-07-14T19:28:04.760323+0000","last_clean":"2023-07-14T19:28:04.760323+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:28:04.760323+0000","last_undegraded":"2023-07-14T19:28:04.760323+0000","last_fullsized":"2023-07-14T19:28:04.760323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:31.761093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952474+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:05.952474+0000","last_peered":"2023-07-14T19:27:05.952474+0000","last_clean":"2023-07-14T19:27:05.952474+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:05.952474+0000","last_undegraded":"2023-07-14T19:27:05.952474+0000","last_fullsized":"2023-07-14T19:27:05.952474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:41:09.562934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768566+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:28:04.768566+0000","last_peered":"2023-07-14T19:28:04.768566+0000","last_clean":"2023-07-14T19:28:04.768566+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:28:04.768566+0000","last_undegraded":"2023-07-14T19:28:04.768566+0000","last_fullsized":"2023-07-14T19:28:04.768566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:37:51.070907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952595+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:27:05.952595+0000","last_peered":"2023-07-14T19:27:05.952595+0000","last_clean":"2023-07-14T19:27:05.952595+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:27:05.952595+0000","last_undegraded":"2023-07-14T19:27:05.952595+0000","last_fullsized":"2023-07-14T19:27:05.952595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:10.381760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763772+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:28:04.763772+0000","last_peered":"2023-07-14T19:28:04.763772+0000","last_clean":"2023-07-14T19:28:04.763772+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:28:04.763772+0000","last_undegraded":"2023-07-14T19:28:04.763772+0000","last_fullsized":"2023-07-14T19:28:04.763772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:10.272127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":214,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:20.654387+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:28:20.654387+0000","last_peered":"2023-07-14T19:28:20.654387+0000","last_clean":"2023-07-14T19:28:20.654387+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:28:20.654387+0000","last_undegraded":"2023-07-14T19:28:20.654387+0000","last_fullsized":"2023-07-14T19:28:20.654387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:43.292916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56889,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":138,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756776+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:28:04.756776+0000","last_peered":"2023-07-14T19:28:04.756776+0000","last_clean":"2023-07-14T19:28:04.756776+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:28:04.756776+0000","last_undegraded":"2023-07-14T19:28:04.756776+0000","last_fullsized":"2023-07-14T19:28:04.756776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:55.206353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.804102+0000","last_change":"2023-07-14T19:27:05.803684+0000","last_active":"2023-07-14T19:27:05.804102+0000","last_peered":"2023-07-14T19:27:05.804102+0000","last_clean":"2023-07-14T19:27:05.804102+0000","last_became_active":"2023-07-14T19:27:05.796450+0000","last_became_peered":"2023-07-14T19:27:05.796450+0000","last_unstale":"2023-07-14T19:27:05.804102+0000","last_undegraded":"2023-07-14T19:27:05.804102+0000","last_fullsized":"2023-07-14T19:27:05.804102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:01.367033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.755135+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:28:04.755135+0000","last_peered":"2023-07-14T19:28:04.755135+0000","last_clean":"2023-07-14T19:28:04.755135+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:28:04.755135+0000","last_undegraded":"2023-07-14T19:28:04.755135+0000","last_fullsized":"2023-07-14T19:28:04.755135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:45:05.169932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952203+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:05.952203+0000","last_peered":"2023-07-14T19:27:05.952203+0000","last_clean":"2023-07-14T19:27:05.952203+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:05.952203+0000","last_undegraded":"2023-07-14T19:27:05.952203+0000","last_fullsized":"2023-07-14T19:27:05.952203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:08:51.150447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760213+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:28:04.760213+0000","last_peered":"2023-07-14T19:28:04.760213+0000","last_clean":"2023-07-14T19:28:04.760213+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:28:04.760213+0000","last_undegraded":"2023-07-14T19:28:04.760213+0000","last_fullsized":"2023-07-14T19:28:04.760213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:01:15.875961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":104,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.654308+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:28:19.654308+0000","last_peered":"2023-07-14T19:28:19.654308+0000","last_clean":"2023-07-14T19:28:19.654308+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:28:19.654308+0000","last_undegraded":"2023-07-14T19:28:19.654308+0000","last_fullsized":"2023-07-14T19:28:19.654308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:27.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764716+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:28:04.764716+0000","last_peered":"2023-07-14T19:28:04.764716+0000","last_clean":"2023-07-14T19:28:04.764716+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:28:04.764716+0000","last_undegraded":"2023-07-14T19:28:04.764716+0000","last_fullsized":"2023-07-14T19:28:04.764716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:10.265120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795312+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:05.795312+0000","last_peered":"2023-07-14T19:27:05.795312+0000","last_clean":"2023-07-14T19:27:05.795312+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:05.795312+0000","last_undegraded":"2023-07-14T19:27:05.795312+0000","last_fullsized":"2023-07-14T19:27:05.795312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:17:12.935696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763373+0000","last_change":"2023-07-14T19:27:05.906040+0000","last_active":"2023-07-14T19:28:04.763373+0000","last_peered":"2023-07-14T19:28:04.763373+0000","last_clean":"2023-07-14T19:28:04.763373+0000","last_became_active":"2023-07-14T19:27:05.905661+0000","last_became_peered":"2023-07-14T19:27:05.905661+0000","last_unstale":"2023-07-14T19:28:04.763373+0000","last_undegraded":"2023-07-14T19:28:04.763373+0000","last_fullsized":"2023-07-14T19:28:04.763373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:29.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795885+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:27:05.795885+0000","last_peered":"2023-07-14T19:27:05.795885+0000","last_clean":"2023-07-14T19:27:05.795885+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:27:05.795885+0000","last_undegraded":"2023-07-14T19:27:05.795885+0000","last_fullsized":"2023-07-14T19:27:05.795885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:46.077263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758100+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:28:04.758100+0000","last_peered":"2023-07-14T19:28:04.758100+0000","last_clean":"2023-07-14T19:28:04.758100+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:28:04.758100+0000","last_undegraded":"2023-07-14T19:28:04.758100+0000","last_fullsized":"2023-07-14T19:28:04.758100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:30.860044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.790247+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:27:05.790247+0000","last_peered":"2023-07-14T19:27:05.790247+0000","last_clean":"2023-07-14T19:27:05.790247+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:27:05.790247+0000","last_undegraded":"2023-07-14T19:27:05.790247+0000","last_fullsized":"2023-07-14T19:27:05.790247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:44:21.663771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758694+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:28:04.758694+0000","last_peered":"2023-07-14T19:28:04.758694+0000","last_clean":"2023-07-14T19:28:04.758694+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:28:04.758694+0000","last_undegraded":"2023-07-14T19:28:04.758694+0000","last_fullsized":"2023-07-14T19:28:04.758694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:38:26.970479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952264+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:27:05.952264+0000","last_peered":"2023-07-14T19:27:05.952264+0000","last_clean":"2023-07-14T19:27:05.952264+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:27:05.952264+0000","last_undegraded":"2023-07-14T19:27:05.952264+0000","last_fullsized":"2023-07-14T19:27:05.952264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:28.081451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769444+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:28:04.769444+0000","last_peered":"2023-07-14T19:28:04.769444+0000","last_clean":"2023-07-14T19:28:04.769444+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:28:04.769444+0000","last_undegraded":"2023-07-14T19:28:04.769444+0000","last_fullsized":"2023-07-14T19:28:04.769444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:40.592640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.752609+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:28:04.752609+0000","last_peered":"2023-07-14T19:28:04.752609+0000","last_clean":"2023-07-14T19:28:04.752609+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:28:04.752609+0000","last_undegraded":"2023-07-14T19:28:04.752609+0000","last_fullsized":"2023-07-14T19:28:04.752609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:29.089446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:20.654935+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:28:20.654935+0000","last_peered":"2023-07-14T19:28:20.654935+0000","last_clean":"2023-07-14T19:28:20.654935+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:28:20.654935+0000","last_undegraded":"2023-07-14T19:28:20.654935+0000","last_fullsized":"2023-07-14T19:28:20.654935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:52:38.820059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765118+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:28:04.765118+0000","last_peered":"2023-07-14T19:28:04.765118+0000","last_clean":"2023-07-14T19:28:04.765118+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:28:04.765118+0000","last_undegraded":"2023-07-14T19:28:04.765118+0000","last_fullsized":"2023-07-14T19:28:04.765118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:20.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789079+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:27:05.789079+0000","last_peered":"2023-07-14T19:27:05.789079+0000","last_clean":"2023-07-14T19:27:05.789079+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:27:05.789079+0000","last_undegraded":"2023-07-14T19:27:05.789079+0000","last_fullsized":"2023-07-14T19:27:05.789079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:11.895409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952086+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:27:05.952086+0000","last_peered":"2023-07-14T19:27:05.952086+0000","last_clean":"2023-07-14T19:27:05.952086+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:27:05.952086+0000","last_undegraded":"2023-07-14T19:27:05.952086+0000","last_fullsized":"2023-07-14T19:27:05.952086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:57.503183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756998+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:28:04.756998+0000","last_peered":"2023-07-14T19:28:04.756998+0000","last_clean":"2023-07-14T19:28:04.756998+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:28:04.756998+0000","last_undegraded":"2023-07-14T19:28:04.756998+0000","last_fullsized":"2023-07-14T19:28:04.756998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:37.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952256+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:05.952256+0000","last_peered":"2023-07-14T19:27:05.952256+0000","last_clean":"2023-07-14T19:27:05.952256+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:05.952256+0000","last_undegraded":"2023-07-14T19:27:05.952256+0000","last_fullsized":"2023-07-14T19:27:05.952256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.155552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795661+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:05.795661+0000","last_peered":"2023-07-14T19:27:05.795661+0000","last_clean":"2023-07-14T19:27:05.795661+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:05.795661+0000","last_undegraded":"2023-07-14T19:27:05.795661+0000","last_fullsized":"2023-07-14T19:27:05.795661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:05.271093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760036+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:28:04.760036+0000","last_peered":"2023-07-14T19:28:04.760036+0000","last_clean":"2023-07-14T19:28:04.760036+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:28:04.760036+0000","last_undegraded":"2023-07-14T19:28:04.760036+0000","last_fullsized":"2023-07-14T19:28:04.760036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:12:18.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:10.903427+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:27:10.903427+0000","last_peered":"2023-07-14T19:27:10.903427+0000","last_clean":"2023-07-14T19:27:10.903427+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:27:10.903427+0000","last_undegraded":"2023-07-14T19:27:10.903427+0000","last_fullsized":"2023-07-14T19:27:10.903427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:26.769682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762774+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:28:04.762774+0000","last_peered":"2023-07-14T19:28:04.762774+0000","last_clean":"2023-07-14T19:28:04.762774+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:28:04.762774+0000","last_undegraded":"2023-07-14T19:28:04.762774+0000","last_fullsized":"2023-07-14T19:28:04.762774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:14.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788996+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:27:05.788996+0000","last_peered":"2023-07-14T19:27:05.788996+0000","last_clean":"2023-07-14T19:27:05.788996+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:27:05.788996+0000","last_undegraded":"2023-07-14T19:27:05.788996+0000","last_fullsized":"2023-07-14T19:27:05.788996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:04:44.601003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.767489+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:28:04.767489+0000","last_peered":"2023-07-14T19:28:04.767489+0000","last_clean":"2023-07-14T19:28:04.767489+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:28:04.767489+0000","last_undegraded":"2023-07-14T19:28:04.767489+0000","last_fullsized":"2023-07-14T19:28:04.767489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:45:38.640268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903756+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:05.903756+0000","last_peered":"2023-07-14T19:27:05.903756+0000","last_clean":"2023-07-14T19:27:05.903756+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:05.903756+0000","last_undegraded":"2023-07-14T19:27:05.903756+0000","last_fullsized":"2023-07-14T19:27:05.903756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:31.831826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.739189+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:28:04.739189+0000","last_peered":"2023-07-14T19:28:04.739189+0000","last_clean":"2023-07-14T19:28:04.739189+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:28:04.739189+0000","last_undegraded":"2023-07-14T19:28:04.739189+0000","last_fullsized":"2023-07-14T19:28:04.739189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:50.824483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787960+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:05.787960+0000","last_peered":"2023-07-14T19:27:05.787960+0000","last_clean":"2023-07-14T19:27:05.787960+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:05.787960+0000","last_undegraded":"2023-07-14T19:27:05.787960+0000","last_fullsized":"2023-07-14T19:27:05.787960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:47.286623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764282+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:28:04.764282+0000","last_peered":"2023-07-14T19:28:04.764282+0000","last_clean":"2023-07-14T19:28:04.764282+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:28:04.764282+0000","last_undegraded":"2023-07-14T19:28:04.764282+0000","last_fullsized":"2023-07-14T19:28:04.764282+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:20.246759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.806328+0000","last_change":"2023-07-14T19:27:05.805846+0000","last_active":"2023-07-14T19:27:05.806328+0000","last_peered":"2023-07-14T19:27:05.806328+0000","last_clean":"2023-07-14T19:27:05.806328+0000","last_became_active":"2023-07-14T19:27:05.796805+0000","last_became_peered":"2023-07-14T19:27:05.796805+0000","last_unstale":"2023-07-14T19:27:05.806328+0000","last_undegraded":"2023-07-14T19:27:05.806328+0000","last_fullsized":"2023-07-14T19:27:05.806328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:47:58.264439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.771308+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:28:04.771308+0000","last_peered":"2023-07-14T19:28:04.771308+0000","last_clean":"2023-07-14T19:28:04.771308+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:28:04.771308+0000","last_undegraded":"2023-07-14T19:28:04.771308+0000","last_fullsized":"2023-07-14T19:28:04.771308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:54:50.760237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795490+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:05.795490+0000","last_peered":"2023-07-14T19:27:05.795490+0000","last_clean":"2023-07-14T19:27:05.795490+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:05.795490+0000","last_undegraded":"2023-07-14T19:27:05.795490+0000","last_fullsized":"2023-07-14T19:27:05.795490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:10:28.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760928+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:28:04.760928+0000","last_peered":"2023-07-14T19:28:04.760928+0000","last_clean":"2023-07-14T19:28:04.760928+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:28:04.760928+0000","last_undegraded":"2023-07-14T19:28:04.760928+0000","last_fullsized":"2023-07-14T19:28:04.760928+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:47:05.779967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795623+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:05.795623+0000","last_peered":"2023-07-14T19:27:05.795623+0000","last_clean":"2023-07-14T19:27:05.795623+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:05.795623+0000","last_undegraded":"2023-07-14T19:27:05.795623+0000","last_fullsized":"2023-07-14T19:27:05.795623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:42.189785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766892+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:28:04.766892+0000","last_peered":"2023-07-14T19:28:04.766892+0000","last_clean":"2023-07-14T19:28:04.766892+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:28:04.766892+0000","last_undegraded":"2023-07-14T19:28:04.766892+0000","last_fullsized":"2023-07-14T19:28:04.766892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:31:52.615160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952458+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:27:05.952458+0000","last_peered":"2023-07-14T19:27:05.952458+0000","last_clean":"2023-07-14T19:27:05.952458+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:27:05.952458+0000","last_undegraded":"2023-07-14T19:27:05.952458+0000","last_fullsized":"2023-07-14T19:27:05.952458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:34.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764119+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:28:04.764119+0000","last_peered":"2023-07-14T19:28:04.764119+0000","last_clean":"2023-07-14T19:28:04.764119+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:28:04.764119+0000","last_undegraded":"2023-07-14T19:28:04.764119+0000","last_fullsized":"2023-07-14T19:28:04.764119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:50:19.375696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951776+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:27:05.951776+0000","last_peered":"2023-07-14T19:27:05.951776+0000","last_clean":"2023-07-14T19:27:05.951776+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:27:05.951776+0000","last_undegraded":"2023-07-14T19:27:05.951776+0000","last_fullsized":"2023-07-14T19:27:05.951776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:55.101835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758103+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:28:04.758103+0000","last_peered":"2023-07-14T19:28:04.758103+0000","last_clean":"2023-07-14T19:28:04.758103+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:28:04.758103+0000","last_undegraded":"2023-07-14T19:28:04.758103+0000","last_fullsized":"2023-07-14T19:28:04.758103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:56:02.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762361+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:28:04.762361+0000","last_peered":"2023-07-14T19:28:04.762361+0000","last_clean":"2023-07-14T19:28:04.762361+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:28:04.762361+0000","last_undegraded":"2023-07-14T19:28:04.762361+0000","last_fullsized":"2023-07-14T19:28:04.762361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:03:59.337396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759168+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:28:04.759168+0000","last_peered":"2023-07-14T19:28:04.759168+0000","last_clean":"2023-07-14T19:28:04.759168+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:28:04.759168+0000","last_undegraded":"2023-07-14T19:28:04.759168+0000","last_fullsized":"2023-07-14T19:28:04.759168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:26:45.117785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763076+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:28:04.763076+0000","last_peered":"2023-07-14T19:28:04.763076+0000","last_clean":"2023-07-14T19:28:04.763076+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:28:04.763076+0000","last_undegraded":"2023-07-14T19:28:04.763076+0000","last_fullsized":"2023-07-14T19:28:04.763076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:59.431251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.773029+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:28:04.773029+0000","last_peered":"2023-07-14T19:28:04.773029+0000","last_clean":"2023-07-14T19:28:04.773029+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:28:04.773029+0000","last_undegraded":"2023-07-14T19:28:04.773029+0000","last_fullsized":"2023-07-14T19:28:04.773029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.135050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764144+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:28:04.764144+0000","last_peered":"2023-07-14T19:28:04.764144+0000","last_clean":"2023-07-14T19:28:04.764144+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:28:04.764144+0000","last_undegraded":"2023-07-14T19:28:04.764144+0000","last_fullsized":"2023-07-14T19:28:04.764144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:24.075259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760086+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:28:04.760086+0000","last_peered":"2023-07-14T19:28:04.760086+0000","last_clean":"2023-07-14T19:28:04.760086+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:28:04.760086+0000","last_undegraded":"2023-07-14T19:28:04.760086+0000","last_fullsized":"2023-07-14T19:28:04.760086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:50.553918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.750630+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:28:04.750630+0000","last_peered":"2023-07-14T19:28:04.750630+0000","last_clean":"2023-07-14T19:28:04.750630+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:28:04.750630+0000","last_undegraded":"2023-07-14T19:28:04.750630+0000","last_fullsized":"2023-07-14T19:28:04.750630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:43:45.456973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756375+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:28:04.756375+0000","last_peered":"2023-07-14T19:28:04.756375+0000","last_clean":"2023-07-14T19:28:04.756375+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:28:04.756375+0000","last_undegraded":"2023-07-14T19:28:04.756375+0000","last_fullsized":"2023-07-14T19:28:04.756375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:43.308371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.761507+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:28:04.761507+0000","last_peered":"2023-07-14T19:28:04.761507+0000","last_clean":"2023-07-14T19:28:04.761507+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:28:04.761507+0000","last_undegraded":"2023-07-14T19:28:04.761507+0000","last_fullsized":"2023-07-14T19:28:04.761507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:02:45.526660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759622+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:28:04.759622+0000","last_peered":"2023-07-14T19:28:04.759622+0000","last_clean":"2023-07-14T19:28:04.759622+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:28:04.759622+0000","last_undegraded":"2023-07-14T19:28:04.759622+0000","last_fullsized":"2023-07-14T19:28:04.759622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:50.377751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765248+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:28:04.765248+0000","last_peered":"2023-07-14T19:28:04.765248+0000","last_clean":"2023-07-14T19:28:04.765248+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:28:04.765248+0000","last_undegraded":"2023-07-14T19:28:04.765248+0000","last_fullsized":"2023-07-14T19:28:04.765248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:51:09.749154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769059+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:28:04.769059+0000","last_peered":"2023-07-14T19:28:04.769059+0000","last_clean":"2023-07-14T19:28:04.769059+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:28:04.769059+0000","last_undegraded":"2023-07-14T19:28:04.769059+0000","last_fullsized":"2023-07-14T19:28:04.769059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:02.143813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759931+0000","last_change":"2023-07-14T19:27:05.796530+0000","last_active":"2023-07-14T19:28:04.759931+0000","last_peered":"2023-07-14T19:28:04.759931+0000","last_clean":"2023-07-14T19:28:04.759931+0000","last_became_active":"2023-07-14T19:27:05.796148+0000","last_became_peered":"2023-07-14T19:27:05.796148+0000","last_unstale":"2023-07-14T19:28:04.759931+0000","last_undegraded":"2023-07-14T19:28:04.759931+0000","last_fullsized":"2023-07-14T19:28:04.759931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:52.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61252,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":505,"num_read_kb":318,"num_write":157,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":192078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514525,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69499999999999995,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.499},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.39000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":5.8380000000000001,"15min":5.8380000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.79300000000000004,"5min":0.874,"15min":0.874},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Jul 14 19:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.53000000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.56499999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.87,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.88700000000000001,"5min":1.179,"15min":1.179},"last":0.86699999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.90500000000000003}]},{"osd":9,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89900000000000002,"15min":0.91300000000000003},"min":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96599999999999997,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.98199999999999998,"15min":1.0149999999999999},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.89400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.998,"5min":1.083,"15min":1.083},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.82799999999999996,"15min":0.82299999999999995},"min":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.016,"5min":1.375,"15min":1.375},"last":0.88}]}]},{"osd":10,"up_from":69,"seq":296352743457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1080420,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.47099999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.69799999999999995,"15min":0.72299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":0.998,"15min":0.998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.66400000000000003,"15min":0.68400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70299999999999996,"5min":1.222,"15min":1.222},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67000000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78200000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.879,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.5,"15min":0.5},"max":{"1min":0.89700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82399999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83599999999999997,"5min":1.24,"15min":1.24},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79700000000000004,"15min":0.80500000000000005},"min":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88200000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.874},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95799999999999996,"5min":1.095,"15min":1.095},"last":0.72899999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.82399999999999995,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.88100000000000001},"min":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93000000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Jul 14 19:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.93100000000000005,"5min":1.02,"15min":1.02},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.627,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.94599999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.89200000000000002}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":432010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57499999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55600000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.438}]},{"osd":1,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79000000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77900000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.70699999999999996,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72199999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67700000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67100000000000004,"5min":0.753,"15min":0.753},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76300000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89200000000000002,"5min":1.081,"15min":1.081},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":1.087,"15min":1.087},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.74299999999999999,"15min":0.76200000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81999999999999995,"5min":1.141,"15min":1.141},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83499999999999996,"5min":1.367,"15min":1.367},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 14 19:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.90300000000000002,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.86599999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.70399999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":431930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64000000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.5,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.504}]},{"osd":3,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.751,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88700000000000001,"5min":1.534,"15min":1.534},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.628},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86599999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95599999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.675,"15min":1.675},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.93500000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.56699999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.627,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.75,"15min":0.72599999999999998},"min":{"1min":0.72499999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0189999999999999,"5min":1.03,"15min":1.03},"last":0.99299999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83299999999999996}]}]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1023579,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.5,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.502,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84799999999999998,"5min":1.024,"15min":1.024},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.47399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82599999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92500000000000004,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.52600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97499999999999998,"5min":1.127,"15min":1.127},"last":0.86399999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.91400000000000003,"5min":1.954,"15min":1.954},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.73699999999999999,"15min":0.70599999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.878,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93000000000000005,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":1,"15min":1},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.76900000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.7610000000000001,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.93899999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94599999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91100000000000003}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.98899999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":431405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.374},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.54200000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":1.212,"15min":1.212},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.63800000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56100000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75600000000000001,"5min":1.115,"15min":1.115},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.032,"15min":1.032},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70399999999999996,"15min":0.72199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.077,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93100000000000005,"5min":1.151,"15min":1.151},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.68500000000000005,"15min":0.72199999999999998},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.68200000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.68799999999999994,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.59999999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69699999999999995,"15min":0.71999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3480000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67600000000000005,"5min":1.55,"15min":1.55},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.746,"15min":0.76400000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":492401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.36199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57799999999999996,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.876},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57199999999999995,"15min":0.51200000000000001},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84699999999999998,"5min":1.302,"15min":1.302},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.41899999999999998},"min":{"1min":0.41199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.626},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.48999999999999999,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63100000000000001,"15min":0.58499999999999996},"min":{"1min":0.51200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.627,"15min":0.57899999999999996},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71899999999999997,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.60699999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.879,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.58699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.98499999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.69199999999999995,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67900000000000005,"15min":0.63900000000000001},"min":{"1min":0.60599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.66000000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.93600000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":1.087,"15min":1.087},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.65700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.996}]},{"osd":11,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":1.042},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.0509999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":432670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.48699999999999999},"min":{"1min":0.34999999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.874,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.749,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.55400000000000005},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.376},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.435}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71199999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76900000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77300000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65600000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.77600000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":494200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.47599999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.55200000000000005,"15min":0.48699999999999999},"min":{"1min":0.40799999999999997,"5min":0.316,"15min":0.316},"max":{"1min":3.8759999999999999,"5min":3.8759999999999999,"15min":3.8759999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57499999999999996,"15min":0.51800000000000002},"min":{"1min":0.40699999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58699999999999997,"15min":0.52400000000000002},"min":{"1min":0.375,"5min":0.255,"15min":0.255},"max":{"1min":0.83899999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60799999999999998,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.436},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.9359999999999999,"5min":4.9359999999999999,"15min":4.9359999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.45100000000000001},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.56100000000000005,"15min":0.52500000000000002},"min":{"1min":0.52400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.7560000000000002,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.753},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.69699999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.65400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.754,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149757,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":431722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32100000000000001},"min":{"1min":0.28299999999999997,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55300000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68000000000000005,"5min":1.202,"15min":1.202},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.503},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.46100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.54300000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.442},{"interface":"front","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.373}]},{"osd":7,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78800000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.71199999999999997,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.006,"15min":1.006},"last":0.66500000000000004}]},{"osd":11,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67500000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.39300000000000002,"15min":0.372},"min":{"1min":0.32200000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.81000000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.52800000000000002},"min":{"1min":0.44700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.41399999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58199999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85899999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61099999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1023311,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.68200000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.996,"15min":0.996},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.42799999999999999},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":1.022,"15min":1.022},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77100000000000002,"5min":1.044,"15min":1.044},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64700000000000002,"15min":0.628},"min":{"1min":0.58799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.624,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91700000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81299999999999994,"5min":1.087,"15min":1.087},"last":0.749},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":1.113,"15min":1.113},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Jul 14 19:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95299999999999996,"5min":1.004,"15min":1.004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.59399999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.91400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.46700000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93799999999999994,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.57899999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.875}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Jul 14 19:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.73699999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.753,"15min":0.747},"min":{"1min":0.64600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:28:23.778 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-14T19:28:23.779 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-14T19:28:23.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 -- 172.21.15.82:0/257500692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0fee90 msgr2=0x7fe35c0ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 --2- 172.21.15.82:0/257500692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0fee90 0x7fe35c0ff2e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe350009990 tx=0x7fe350009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 -- 172.21.15.82:0/257500692 shutdown_connections 2023-07-14T19:28:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 --2- 172.21.15.82:0/257500692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c0ff820 0x7fe35c1070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 --2- 172.21.15.82:0/257500692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c100160 0x7fe35c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 --2- 172.21.15.82:0/257500692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0fee90 0x7fe35c0ff2e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.930+0000 7fe36188a700 1 -- 172.21.15.82:0/257500692 >> 172.21.15.82:0/257500692 conn(0x7fe35c0f9990 msgr2=0x7fe35c0fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:23.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 -- 172.21.15.82:0/257500692 shutdown_connections 2023-07-14T19:28:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 -- 172.21.15.82:0/257500692 wait complete. 2023-07-14T19:28:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 Processor -- start 2023-07-14T19:28:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 -- start start 2023-07-14T19:28:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c0fee90 0x7fe35c0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 0x7fe35c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c100160 0x7fe35c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d400 con 0x7fe35c0ff820 2023-07-14T19:28:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d570 con 0x7fe35c100160 2023-07-14T19:28:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.931+0000 7fe36188a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c19d730 con 0x7fe35c0fee90 2023-07-14T19:28:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 0x7fe35c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 0x7fe35c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42202/0 (socket says 172.21.15.82:42202) 2023-07-14T19:28:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 -- 172.21.15.82:0/2265309122 learned_addr learned my addr 172.21.15.82:0/2265309122 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35affd700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c0fee90 0x7fe35c0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35b7fe700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c100160 0x7fe35c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 -- 172.21.15.82:0/2265309122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c0fee90 msgr2=0x7fe35c0fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.939 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c0fee90 0x7fe35c0fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 -- 172.21.15.82:0/2265309122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c100160 msgr2=0x7fe35c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c100160 0x7fe35c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 -- 172.21.15.82:0/2265309122 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500095f0 con 0x7fe35c0ff820 2023-07-14T19:28:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe35a7fc700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 0x7fe35c19a620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe3500059d0 tx=0x7fe3500047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:23.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35001d070 con 0x7fe35c0ff820 2023-07-14T19:28:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c19d9b0 con 0x7fe35c0ff820 2023-07-14T19:28:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35000fb90 con 0x7fe35c0ff820 2023-07-14T19:28:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350017650 con 0x7fe35c0ff820 2023-07-14T19:28:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.932+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c19ddd0 con 0x7fe35c0ff820 2023-07-14T19:28:23.941 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.933+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c106220 con 0x7fe35c0ff820 2023-07-14T19:28:23.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.934+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe35000fd00 con 0x7fe35c0ff820 2023-07-14T19:28:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.935+0000 7fe360888700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3440a11a0 0x7fe3440a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.935+0000 7fe35affd700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3440a11a0 0x7fe3440a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.935+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fe3500cdf10 con 0x7fe35c0ff820 2023-07-14T19:28:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.935+0000 7fe35affd700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3440a11a0 0x7fe3440a3650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0687c0 tx=0x7fe34c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:23.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:23.939+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fe3500977b0 con 0x7fe35c0ff820 2023-07-14T19:28:24.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.125+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe35c060a80 con 0x7fe3440a11a0 2023-07-14T19:28:24.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.135+0000 7fe360888700 1 -- 172.21.15.82:0/2265309122 <== mgr.14152 v2:172.21.15.82:6800/504215117 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344000 (secure 0 0 0) 0x7fe35c060a80 con 0x7fe3440a11a0 2023-07-14T19:28:24.140 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:24.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3440a11a0 msgr2=0x7fe3440a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3440a11a0 0x7fe3440a3650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0687c0 tx=0x7fe34c009260 comp rx=0 tx=0).stop 2023-07-14T19:28:24.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 msgr2=0x7fe35c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 0x7fe35c19a620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe3500059d0 tx=0x7fe3500047a0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 shutdown_connections 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe35c100160 0x7fe35c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe35c0fee90 0x7fe35c0fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fe3440a11a0 0x7fe3440a3650 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 --2- 172.21.15.82:0/2265309122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe35c0ff820 0x7fe35c19a620 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 >> 172.21.15.82:0/2265309122 conn(0x7fe35c0f9990 msgr2=0x7fe35c103360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 shutdown_connections 2023-07-14T19:28:24.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.139+0000 7fe36188a700 1 -- 172.21.15.82:0/2265309122 wait complete. 2023-07-14T19:28:24.144 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2023-07-14T19:28:24.151 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:23 smithi082 ceph-mon[127461]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T19:28:24.151 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:23 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1643966330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:24.152 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-07-14T19:28:22.738141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651866,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":641,"num_read_kb":437,"num_write":340,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":777,"ondisk_log_size":777,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335172,"kb_used_data":10980,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606652,"statfs":{"total":1151940427776,"available":1151597211648,"internally_reserved":0,"allocated":11243520,"data_stored":7137562,"data_compressed":107211,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":47,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766460+0000","last_change":"2023-07-14T19:26:57.324923+0000","last_active":"2023-07-14T19:28:04.766460+0000","last_peered":"2023-07-14T19:28:04.766460+0000","last_clean":"2023-07-14T19:28:04.766460+0000","last_became_active":"2023-07-14T19:26:57.324642+0000","last_became_peered":"2023-07-14T19:26:57.324642+0000","last_unstale":"2023-07-14T19:28:04.766460+0000","last_undegraded":"2023-07-14T19:28:04.766460+0000","last_fullsized":"2023-07-14T19:28:04.766460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:28:41.471339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769021+0000","last_change":"2023-07-14T19:26:57.323210+0000","last_active":"2023-07-14T19:28:04.769021+0000","last_peered":"2023-07-14T19:28:04.769021+0000","last_clean":"2023-07-14T19:28:04.769021+0000","last_became_active":"2023-07-14T19:26:57.322895+0000","last_became_peered":"2023-07-14T19:26:57.322895+0000","last_unstale":"2023-07-14T19:28:04.769021+0000","last_undegraded":"2023-07-14T19:28:04.769021+0000","last_fullsized":"2023-07-14T19:28:04.769021+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:01.099918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765502+0000","last_change":"2023-07-14T19:26:57.326860+0000","last_active":"2023-07-14T19:28:04.765502+0000","last_peered":"2023-07-14T19:28:04.765502+0000","last_clean":"2023-07-14T19:28:04.765502+0000","last_became_active":"2023-07-14T19:26:57.326589+0000","last_became_peered":"2023-07-14T19:26:57.326589+0000","last_unstale":"2023-07-14T19:28:04.765502+0000","last_undegraded":"2023-07-14T19:28:04.765502+0000","last_fullsized":"2023-07-14T19:28:04.765502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:33:44.231618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.761968+0000","last_change":"2023-07-14T19:26:57.322455+0000","last_active":"2023-07-14T19:28:04.761968+0000","last_peered":"2023-07-14T19:28:04.761968+0000","last_clean":"2023-07-14T19:28:04.761968+0000","last_became_active":"2023-07-14T19:26:57.321976+0000","last_became_peered":"2023-07-14T19:26:57.321976+0000","last_unstale":"2023-07-14T19:28:04.761968+0000","last_undegraded":"2023-07-14T19:28:04.761968+0000","last_fullsized":"2023-07-14T19:28:04.761968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:38:02.561344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.757252+0000","last_change":"2023-07-14T19:26:57.313654+0000","last_active":"2023-07-14T19:28:04.757252+0000","last_peered":"2023-07-14T19:28:04.757252+0000","last_clean":"2023-07-14T19:28:04.757252+0000","last_became_active":"2023-07-14T19:26:57.313300+0000","last_became_peered":"2023-07-14T19:26:57.313300+0000","last_unstale":"2023-07-14T19:28:04.757252+0000","last_undegraded":"2023-07-14T19:28:04.757252+0000","last_fullsized":"2023-07-14T19:28:04.757252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:36:44.500967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759197+0000","last_change":"2023-07-14T19:26:57.327636+0000","last_active":"2023-07-14T19:28:04.759197+0000","last_peered":"2023-07-14T19:28:04.759197+0000","last_clean":"2023-07-14T19:28:04.759197+0000","last_became_active":"2023-07-14T19:26:57.327358+0000","last_became_peered":"2023-07-14T19:26:57.327358+0000","last_unstale":"2023-07-14T19:28:04.759197+0000","last_undegraded":"2023-07-14T19:28:04.759197+0000","last_fullsized":"2023-07-14T19:28:04.759197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.697906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764042+0000","last_change":"2023-07-14T19:26:57.315868+0000","last_active":"2023-07-14T19:28:04.764042+0000","last_peered":"2023-07-14T19:28:04.764042+0000","last_clean":"2023-07-14T19:28:04.764042+0000","last_became_active":"2023-07-14T19:26:57.315559+0000","last_became_peered":"2023-07-14T19:26:57.315559+0000","last_unstale":"2023-07-14T19:28:04.764042+0000","last_undegraded":"2023-07-14T19:28:04.764042+0000","last_fullsized":"2023-07-14T19:28:04.764042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.817064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768382+0000","last_change":"2023-07-14T19:26:57.328348+0000","last_active":"2023-07-14T19:28:04.768382+0000","last_peered":"2023-07-14T19:28:04.768382+0000","last_clean":"2023-07-14T19:28:04.768382+0000","last_became_active":"2023-07-14T19:26:57.328037+0000","last_became_peered":"2023-07-14T19:26:57.328037+0000","last_unstale":"2023-07-14T19:28:04.768382+0000","last_undegraded":"2023-07-14T19:28:04.768382+0000","last_fullsized":"2023-07-14T19:28:04.768382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:20:11.270082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765502+0000","last_change":"2023-07-14T19:26:57.325073+0000","last_active":"2023-07-14T19:28:04.765502+0000","last_peered":"2023-07-14T19:28:04.765502+0000","last_clean":"2023-07-14T19:28:04.765502+0000","last_became_active":"2023-07-14T19:26:57.324772+0000","last_became_peered":"2023-07-14T19:26:57.324772+0000","last_unstale":"2023-07-14T19:28:04.765502+0000","last_undegraded":"2023-07-14T19:28:04.765502+0000","last_fullsized":"2023-07-14T19:28:04.765502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:57:14.548210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758142+0000","last_change":"2023-07-14T19:26:57.332413+0000","last_active":"2023-07-14T19:28:04.758142+0000","last_peered":"2023-07-14T19:28:04.758142+0000","last_clean":"2023-07-14T19:28:04.758142+0000","last_became_active":"2023-07-14T19:26:57.331946+0000","last_became_peered":"2023-07-14T19:26:57.331946+0000","last_unstale":"2023-07-14T19:28:04.758142+0000","last_undegraded":"2023-07-14T19:28:04.758142+0000","last_fullsized":"2023-07-14T19:28:04.758142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:21:09.380546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766790+0000","last_change":"2023-07-14T19:26:57.322493+0000","last_active":"2023-07-14T19:28:04.766790+0000","last_peered":"2023-07-14T19:28:04.766790+0000","last_clean":"2023-07-14T19:28:04.766790+0000","last_became_active":"2023-07-14T19:26:57.322150+0000","last_became_peered":"2023-07-14T19:26:57.322150+0000","last_unstale":"2023-07-14T19:28:04.766790+0000","last_undegraded":"2023-07-14T19:28:04.766790+0000","last_fullsized":"2023-07-14T19:28:04.766790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:33:24.877562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763507+0000","last_change":"2023-07-14T19:27:05.797079+0000","last_active":"2023-07-14T19:28:04.763507+0000","last_peered":"2023-07-14T19:28:04.763507+0000","last_clean":"2023-07-14T19:28:04.763507+0000","last_became_active":"2023-07-14T19:27:05.796765+0000","last_became_peered":"2023-07-14T19:27:05.796765+0000","last_unstale":"2023-07-14T19:28:04.763507+0000","last_undegraded":"2023-07-14T19:28:04.763507+0000","last_fullsized":"2023-07-14T19:28:04.763507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:40:18.699215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760253+0000","last_change":"2023-07-14T19:26:57.318837+0000","last_active":"2023-07-14T19:28:04.760253+0000","last_peered":"2023-07-14T19:28:04.760253+0000","last_clean":"2023-07-14T19:28:04.760253+0000","last_became_active":"2023-07-14T19:26:57.318477+0000","last_became_peered":"2023-07-14T19:26:57.318477+0000","last_unstale":"2023-07-14T19:28:04.760253+0000","last_undegraded":"2023-07-14T19:28:04.760253+0000","last_fullsized":"2023-07-14T19:28:04.760253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:16:24.688894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768895+0000","last_change":"2023-07-14T19:26:57.322956+0000","last_active":"2023-07-14T19:28:04.768895+0000","last_peered":"2023-07-14T19:28:04.768895+0000","last_clean":"2023-07-14T19:28:04.768895+0000","last_became_active":"2023-07-14T19:26:57.322593+0000","last_became_peered":"2023-07-14T19:26:57.322593+0000","last_unstale":"2023-07-14T19:28:04.768895+0000","last_undegraded":"2023-07-14T19:28:04.768895+0000","last_fullsized":"2023-07-14T19:28:04.768895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:11:02.752563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768383+0000","last_change":"2023-07-14T19:26:57.326087+0000","last_active":"2023-07-14T19:28:04.768383+0000","last_peered":"2023-07-14T19:28:04.768383+0000","last_clean":"2023-07-14T19:28:04.768383+0000","last_became_active":"2023-07-14T19:26:57.325791+0000","last_became_peered":"2023-07-14T19:26:57.325791+0000","last_unstale":"2023-07-14T19:28:04.768383+0000","last_undegraded":"2023-07-14T19:28:04.768383+0000","last_fullsized":"2023-07-14T19:28:04.768383+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:29.721933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764585+0000","last_change":"2023-07-14T19:26:57.323839+0000","last_active":"2023-07-14T19:28:04.764585+0000","last_peered":"2023-07-14T19:28:04.764585+0000","last_clean":"2023-07-14T19:28:04.764585+0000","last_became_active":"2023-07-14T19:26:57.323518+0000","last_became_peered":"2023-07-14T19:26:57.323518+0000","last_unstale":"2023-07-14T19:28:04.764585+0000","last_undegraded":"2023-07-14T19:28:04.764585+0000","last_fullsized":"2023-07-14T19:28:04.764585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:14.830796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766558+0000","last_change":"2023-07-14T19:26:57.325503+0000","last_active":"2023-07-14T19:28:04.766558+0000","last_peered":"2023-07-14T19:28:04.766558+0000","last_clean":"2023-07-14T19:28:04.766558+0000","last_became_active":"2023-07-14T19:26:57.325086+0000","last_became_peered":"2023-07-14T19:26:57.325086+0000","last_unstale":"2023-07-14T19:28:04.766558+0000","last_undegraded":"2023-07-14T19:28:04.766558+0000","last_fullsized":"2023-07-14T19:28:04.766558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:44:47.623791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762716+0000","last_change":"2023-07-14T19:26:57.321778+0000","last_active":"2023-07-14T19:28:04.762716+0000","last_peered":"2023-07-14T19:28:04.762716+0000","last_clean":"2023-07-14T19:28:04.762716+0000","last_became_active":"2023-07-14T19:26:57.321485+0000","last_became_peered":"2023-07-14T19:26:57.321485+0000","last_unstale":"2023-07-14T19:28:04.762716+0000","last_undegraded":"2023-07-14T19:28:04.762716+0000","last_fullsized":"2023-07-14T19:28:04.762716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:17:45.291302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758438+0000","last_change":"2023-07-14T19:26:57.323909+0000","last_active":"2023-07-14T19:28:04.758438+0000","last_peered":"2023-07-14T19:28:04.758438+0000","last_clean":"2023-07-14T19:28:04.758438+0000","last_became_active":"2023-07-14T19:26:57.323609+0000","last_became_peered":"2023-07-14T19:26:57.323609+0000","last_unstale":"2023-07-14T19:28:04.758438+0000","last_undegraded":"2023-07-14T19:28:04.758438+0000","last_fullsized":"2023-07-14T19:28:04.758438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:34:18.122288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789986+0000","last_change":"2023-07-14T19:26:56.499178+0000","last_active":"2023-07-14T19:27:05.789986+0000","last_peered":"2023-07-14T19:27:05.789986+0000","last_clean":"2023-07-14T19:27:05.789986+0000","last_became_active":"2023-07-14T19:26:56.498860+0000","last_became_peered":"2023-07-14T19:26:56.498860+0000","last_unstale":"2023-07-14T19:27:05.789986+0000","last_undegraded":"2023-07-14T19:27:05.789986+0000","last_fullsized":"2023-07-14T19:27:05.789986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:25:15.693215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762586+0000","last_change":"2023-07-14T19:26:57.316572+0000","last_active":"2023-07-14T19:28:04.762586+0000","last_peered":"2023-07-14T19:28:04.762586+0000","last_clean":"2023-07-14T19:28:04.762586+0000","last_became_active":"2023-07-14T19:26:57.316228+0000","last_became_peered":"2023-07-14T19:26:57.316228+0000","last_unstale":"2023-07-14T19:28:04.762586+0000","last_undegraded":"2023-07-14T19:28:04.762586+0000","last_fullsized":"2023-07-14T19:28:04.762586+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:03:10.138305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788456+0000","last_change":"2023-07-14T19:26:56.499438+0000","last_active":"2023-07-14T19:27:05.788456+0000","last_peered":"2023-07-14T19:27:05.788456+0000","last_clean":"2023-07-14T19:27:05.788456+0000","last_became_active":"2023-07-14T19:26:56.499139+0000","last_became_peered":"2023-07-14T19:26:56.499139+0000","last_unstale":"2023-07-14T19:27:05.788456+0000","last_undegraded":"2023-07-14T19:27:05.788456+0000","last_fullsized":"2023-07-14T19:27:05.788456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:52:33.601716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768924+0000","last_change":"2023-07-14T19:26:57.324644+0000","last_active":"2023-07-14T19:28:04.768924+0000","last_peered":"2023-07-14T19:28:04.768924+0000","last_clean":"2023-07-14T19:28:04.768924+0000","last_became_active":"2023-07-14T19:26:57.324322+0000","last_became_peered":"2023-07-14T19:26:57.324322+0000","last_unstale":"2023-07-14T19:28:04.768924+0000","last_undegraded":"2023-07-14T19:28:04.768924+0000","last_fullsized":"2023-07-14T19:28:04.768924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:47:04.807902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795938+0000","last_change":"2023-07-14T19:26:56.495987+0000","last_active":"2023-07-14T19:27:05.795938+0000","last_peered":"2023-07-14T19:27:05.795938+0000","last_clean":"2023-07-14T19:27:05.795938+0000","last_became_active":"2023-07-14T19:26:56.495661+0000","last_became_peered":"2023-07-14T19:26:56.495661+0000","last_unstale":"2023-07-14T19:27:05.795938+0000","last_undegraded":"2023-07-14T19:27:05.795938+0000","last_fullsized":"2023-07-14T19:27:05.795938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:53:11.673840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.767910+0000","last_change":"2023-07-14T19:26:57.332069+0000","last_active":"2023-07-14T19:28:04.767910+0000","last_peered":"2023-07-14T19:28:04.767910+0000","last_clean":"2023-07-14T19:28:04.767910+0000","last_became_active":"2023-07-14T19:26:57.331750+0000","last_became_peered":"2023-07-14T19:26:57.331750+0000","last_unstale":"2023-07-14T19:28:04.767910+0000","last_undegraded":"2023-07-14T19:28:04.767910+0000","last_fullsized":"2023-07-14T19:28:04.767910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:07:56.838822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795738+0000","last_change":"2023-07-14T19:26:56.499100+0000","last_active":"2023-07-14T19:27:05.795738+0000","last_peered":"2023-07-14T19:27:05.795738+0000","last_clean":"2023-07-14T19:27:05.795738+0000","last_became_active":"2023-07-14T19:26:56.498792+0000","last_became_peered":"2023-07-14T19:26:56.498792+0000","last_unstale":"2023-07-14T19:27:05.795738+0000","last_undegraded":"2023-07-14T19:27:05.795738+0000","last_fullsized":"2023-07-14T19:27:05.795738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:12:12.590934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766461+0000","last_change":"2023-07-14T19:26:57.322329+0000","last_active":"2023-07-14T19:28:04.766461+0000","last_peered":"2023-07-14T19:28:04.766461+0000","last_clean":"2023-07-14T19:28:04.766461+0000","last_became_active":"2023-07-14T19:26:57.322034+0000","last_became_peered":"2023-07-14T19:26:57.322034+0000","last_unstale":"2023-07-14T19:28:04.766461+0000","last_undegraded":"2023-07-14T19:28:04.766461+0000","last_fullsized":"2023-07-14T19:28:04.766461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:21:16.730207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":108,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.652776+0000","last_change":"2023-07-14T19:26:56.502638+0000","last_active":"2023-07-14T19:28:19.652776+0000","last_peered":"2023-07-14T19:28:19.652776+0000","last_clean":"2023-07-14T19:28:19.652776+0000","last_became_active":"2023-07-14T19:26:56.502311+0000","last_became_peered":"2023-07-14T19:26:56.502311+0000","last_unstale":"2023-07-14T19:28:19.652776+0000","last_undegraded":"2023-07-14T19:28:19.652776+0000","last_fullsized":"2023-07-14T19:28:19.652776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:33:07.160367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1105,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759897+0000","last_change":"2023-07-14T19:26:57.313293+0000","last_active":"2023-07-14T19:28:04.759897+0000","last_peered":"2023-07-14T19:28:04.759897+0000","last_clean":"2023-07-14T19:28:04.759897+0000","last_became_active":"2023-07-14T19:26:57.312974+0000","last_became_peered":"2023-07-14T19:26:57.312974+0000","last_unstale":"2023-07-14T19:28:04.759897+0000","last_undegraded":"2023-07-14T19:28:04.759897+0000","last_fullsized":"2023-07-14T19:28:04.759897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:08:02.082831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.794601+0000","last_change":"2023-07-14T19:26:56.495721+0000","last_active":"2023-07-14T19:27:05.794601+0000","last_peered":"2023-07-14T19:27:05.794601+0000","last_clean":"2023-07-14T19:27:05.794601+0000","last_became_active":"2023-07-14T19:26:56.495340+0000","last_became_peered":"2023-07-14T19:26:56.495340+0000","last_unstale":"2023-07-14T19:27:05.794601+0000","last_undegraded":"2023-07-14T19:27:05.794601+0000","last_fullsized":"2023-07-14T19:27:05.794601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:22:58.235627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760570+0000","last_change":"2023-07-14T19:26:57.317263+0000","last_active":"2023-07-14T19:28:04.760570+0000","last_peered":"2023-07-14T19:28:04.760570+0000","last_clean":"2023-07-14T19:28:04.760570+0000","last_became_active":"2023-07-14T19:26:57.316985+0000","last_became_peered":"2023-07-14T19:26:57.316985+0000","last_unstale":"2023-07-14T19:28:04.760570+0000","last_undegraded":"2023-07-14T19:28:04.760570+0000","last_fullsized":"2023-07-14T19:28:04.760570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:07:02.114404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951680+0000","last_change":"2023-07-14T19:26:56.499997+0000","last_active":"2023-07-14T19:27:05.951680+0000","last_peered":"2023-07-14T19:27:05.951680+0000","last_clean":"2023-07-14T19:27:05.951680+0000","last_became_active":"2023-07-14T19:26:56.499713+0000","last_became_peered":"2023-07-14T19:26:56.499713+0000","last_unstale":"2023-07-14T19:27:05.951680+0000","last_undegraded":"2023-07-14T19:27:05.951680+0000","last_fullsized":"2023-07-14T19:27:05.951680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:56:15.018579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758010+0000","last_change":"2023-07-14T19:26:57.325231+0000","last_active":"2023-07-14T19:28:04.758010+0000","last_peered":"2023-07-14T19:28:04.758010+0000","last_clean":"2023-07-14T19:28:04.758010+0000","last_became_active":"2023-07-14T19:26:57.324974+0000","last_became_peered":"2023-07-14T19:26:57.324974+0000","last_unstale":"2023-07-14T19:28:04.758010+0000","last_undegraded":"2023-07-14T19:28:04.758010+0000","last_fullsized":"2023-07-14T19:28:04.758010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:01:26.867144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951987+0000","last_change":"2023-07-14T19:26:56.500581+0000","last_active":"2023-07-14T19:27:05.951987+0000","last_peered":"2023-07-14T19:27:05.951987+0000","last_clean":"2023-07-14T19:27:05.951987+0000","last_became_active":"2023-07-14T19:26:56.500251+0000","last_became_peered":"2023-07-14T19:26:56.500251+0000","last_unstale":"2023-07-14T19:27:05.951987+0000","last_undegraded":"2023-07-14T19:27:05.951987+0000","last_fullsized":"2023-07-14T19:27:05.951987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:50:49.740777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758943+0000","last_change":"2023-07-14T19:26:57.319583+0000","last_active":"2023-07-14T19:28:04.758943+0000","last_peered":"2023-07-14T19:28:04.758943+0000","last_clean":"2023-07-14T19:28:04.758943+0000","last_became_active":"2023-07-14T19:26:57.319271+0000","last_became_peered":"2023-07-14T19:26:57.319271+0000","last_unstale":"2023-07-14T19:28:04.758943+0000","last_undegraded":"2023-07-14T19:28:04.758943+0000","last_fullsized":"2023-07-14T19:28:04.758943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:34:42.105198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":101,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.652227+0000","last_change":"2023-07-14T19:26:56.498997+0000","last_active":"2023-07-14T19:28:19.652227+0000","last_peered":"2023-07-14T19:28:19.652227+0000","last_clean":"2023-07-14T19:28:19.652227+0000","last_became_active":"2023-07-14T19:26:56.498689+0000","last_became_peered":"2023-07-14T19:26:56.498689+0000","last_unstale":"2023-07-14T19:28:19.652227+0000","last_undegraded":"2023-07-14T19:28:19.652227+0000","last_fullsized":"2023-07-14T19:28:19.652227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:16:38.911754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760323+0000","last_change":"2023-07-14T19:26:57.333180+0000","last_active":"2023-07-14T19:28:04.760323+0000","last_peered":"2023-07-14T19:28:04.760323+0000","last_clean":"2023-07-14T19:28:04.760323+0000","last_became_active":"2023-07-14T19:26:57.332908+0000","last_became_peered":"2023-07-14T19:26:57.332908+0000","last_unstale":"2023-07-14T19:28:04.760323+0000","last_undegraded":"2023-07-14T19:28:04.760323+0000","last_fullsized":"2023-07-14T19:28:04.760323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:16:31.761093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952474+0000","last_change":"2023-07-14T19:26:56.495166+0000","last_active":"2023-07-14T19:27:05.952474+0000","last_peered":"2023-07-14T19:27:05.952474+0000","last_clean":"2023-07-14T19:27:05.952474+0000","last_became_active":"2023-07-14T19:26:56.494820+0000","last_became_peered":"2023-07-14T19:26:56.494820+0000","last_unstale":"2023-07-14T19:27:05.952474+0000","last_undegraded":"2023-07-14T19:27:05.952474+0000","last_fullsized":"2023-07-14T19:27:05.952474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:41:09.562934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.768566+0000","last_change":"2023-07-14T19:26:57.325647+0000","last_active":"2023-07-14T19:28:04.768566+0000","last_peered":"2023-07-14T19:28:04.768566+0000","last_clean":"2023-07-14T19:28:04.768566+0000","last_became_active":"2023-07-14T19:26:57.325366+0000","last_became_peered":"2023-07-14T19:26:57.325366+0000","last_unstale":"2023-07-14T19:28:04.768566+0000","last_undegraded":"2023-07-14T19:28:04.768566+0000","last_fullsized":"2023-07-14T19:28:04.768566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:37:51.070907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952595+0000","last_change":"2023-07-14T19:26:56.505136+0000","last_active":"2023-07-14T19:27:05.952595+0000","last_peered":"2023-07-14T19:27:05.952595+0000","last_clean":"2023-07-14T19:27:05.952595+0000","last_became_active":"2023-07-14T19:26:56.504837+0000","last_became_peered":"2023-07-14T19:26:56.504837+0000","last_unstale":"2023-07-14T19:27:05.952595+0000","last_undegraded":"2023-07-14T19:27:05.952595+0000","last_fullsized":"2023-07-14T19:27:05.952595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:00:10.381760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763772+0000","last_change":"2023-07-14T19:26:57.325932+0000","last_active":"2023-07-14T19:28:04.763772+0000","last_peered":"2023-07-14T19:28:04.763772+0000","last_clean":"2023-07-14T19:28:04.763772+0000","last_became_active":"2023-07-14T19:26:57.325619+0000","last_became_peered":"2023-07-14T19:26:57.325619+0000","last_unstale":"2023-07-14T19:28:04.763772+0000","last_undegraded":"2023-07-14T19:28:04.763772+0000","last_fullsized":"2023-07-14T19:28:04.763772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:57:10.272127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":214,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:20.654387+0000","last_change":"2023-07-14T19:26:56.498432+0000","last_active":"2023-07-14T19:28:20.654387+0000","last_peered":"2023-07-14T19:28:20.654387+0000","last_clean":"2023-07-14T19:28:20.654387+0000","last_became_active":"2023-07-14T19:26:56.498163+0000","last_became_peered":"2023-07-14T19:26:56.498163+0000","last_unstale":"2023-07-14T19:28:20.654387+0000","last_undegraded":"2023-07-14T19:28:20.654387+0000","last_fullsized":"2023-07-14T19:28:20.654387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T01:49:43.292916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56889,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":138,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756776+0000","last_change":"2023-07-14T19:26:57.318609+0000","last_active":"2023-07-14T19:28:04.756776+0000","last_peered":"2023-07-14T19:28:04.756776+0000","last_clean":"2023-07-14T19:28:04.756776+0000","last_became_active":"2023-07-14T19:26:57.318102+0000","last_became_peered":"2023-07-14T19:26:57.318102+0000","last_unstale":"2023-07-14T19:28:04.756776+0000","last_undegraded":"2023-07-14T19:28:04.756776+0000","last_fullsized":"2023-07-14T19:28:04.756776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:48:55.206353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.804102+0000","last_change":"2023-07-14T19:27:05.803684+0000","last_active":"2023-07-14T19:27:05.804102+0000","last_peered":"2023-07-14T19:27:05.804102+0000","last_clean":"2023-07-14T19:27:05.804102+0000","last_became_active":"2023-07-14T19:27:05.796450+0000","last_became_peered":"2023-07-14T19:27:05.796450+0000","last_unstale":"2023-07-14T19:27:05.804102+0000","last_undegraded":"2023-07-14T19:27:05.804102+0000","last_fullsized":"2023-07-14T19:27:05.804102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:01.367033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.755135+0000","last_change":"2023-07-14T19:26:57.320309+0000","last_active":"2023-07-14T19:28:04.755135+0000","last_peered":"2023-07-14T19:28:04.755135+0000","last_clean":"2023-07-14T19:28:04.755135+0000","last_became_active":"2023-07-14T19:26:57.319882+0000","last_became_peered":"2023-07-14T19:26:57.319882+0000","last_unstale":"2023-07-14T19:28:04.755135+0000","last_undegraded":"2023-07-14T19:28:04.755135+0000","last_fullsized":"2023-07-14T19:28:04.755135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T04:45:05.169932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952203+0000","last_change":"2023-07-14T19:26:56.497203+0000","last_active":"2023-07-14T19:27:05.952203+0000","last_peered":"2023-07-14T19:27:05.952203+0000","last_clean":"2023-07-14T19:27:05.952203+0000","last_became_active":"2023-07-14T19:26:56.496863+0000","last_became_peered":"2023-07-14T19:26:56.496863+0000","last_unstale":"2023-07-14T19:27:05.952203+0000","last_undegraded":"2023-07-14T19:27:05.952203+0000","last_fullsized":"2023-07-14T19:27:05.952203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:08:51.150447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760213+0000","last_change":"2023-07-14T19:26:57.327205+0000","last_active":"2023-07-14T19:28:04.760213+0000","last_peered":"2023-07-14T19:28:04.760213+0000","last_clean":"2023-07-14T19:28:04.760213+0000","last_became_active":"2023-07-14T19:26:57.326903+0000","last_became_peered":"2023-07-14T19:26:57.326903+0000","last_unstale":"2023-07-14T19:28:04.760213+0000","last_undegraded":"2023-07-14T19:28:04.760213+0000","last_fullsized":"2023-07-14T19:28:04.760213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:01:15.875961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'4","reported_seq":104,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:19.654308+0000","last_change":"2023-07-14T19:26:56.495950+0000","last_active":"2023-07-14T19:28:19.654308+0000","last_peered":"2023-07-14T19:28:19.654308+0000","last_clean":"2023-07-14T19:28:19.654308+0000","last_became_active":"2023-07-14T19:26:56.495656+0000","last_became_peered":"2023-07-14T19:26:56.495656+0000","last_unstale":"2023-07-14T19:28:19.654308+0000","last_undegraded":"2023-07-14T19:28:19.654308+0000","last_fullsized":"2023-07-14T19:28:19.654308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:53:27.466377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":858,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764716+0000","last_change":"2023-07-14T19:26:57.321333+0000","last_active":"2023-07-14T19:28:04.764716+0000","last_peered":"2023-07-14T19:28:04.764716+0000","last_clean":"2023-07-14T19:28:04.764716+0000","last_became_active":"2023-07-14T19:26:57.321020+0000","last_became_peered":"2023-07-14T19:26:57.321020+0000","last_unstale":"2023-07-14T19:28:04.764716+0000","last_undegraded":"2023-07-14T19:28:04.764716+0000","last_fullsized":"2023-07-14T19:28:04.764716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:47:10.265120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795312+0000","last_change":"2023-07-14T19:26:56.494682+0000","last_active":"2023-07-14T19:27:05.795312+0000","last_peered":"2023-07-14T19:27:05.795312+0000","last_clean":"2023-07-14T19:27:05.795312+0000","last_became_active":"2023-07-14T19:26:56.494344+0000","last_became_peered":"2023-07-14T19:26:56.494344+0000","last_unstale":"2023-07-14T19:27:05.795312+0000","last_undegraded":"2023-07-14T19:27:05.795312+0000","last_fullsized":"2023-07-14T19:27:05.795312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:17:12.935696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763373+0000","last_change":"2023-07-14T19:27:05.906040+0000","last_active":"2023-07-14T19:28:04.763373+0000","last_peered":"2023-07-14T19:28:04.763373+0000","last_clean":"2023-07-14T19:28:04.763373+0000","last_became_active":"2023-07-14T19:27:05.905661+0000","last_became_peered":"2023-07-14T19:27:05.905661+0000","last_unstale":"2023-07-14T19:28:04.763373+0000","last_undegraded":"2023-07-14T19:28:04.763373+0000","last_fullsized":"2023-07-14T19:28:04.763373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:56:29.656207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795885+0000","last_change":"2023-07-14T19:26:56.499637+0000","last_active":"2023-07-14T19:27:05.795885+0000","last_peered":"2023-07-14T19:27:05.795885+0000","last_clean":"2023-07-14T19:27:05.795885+0000","last_became_active":"2023-07-14T19:26:56.499321+0000","last_became_peered":"2023-07-14T19:26:56.499321+0000","last_unstale":"2023-07-14T19:27:05.795885+0000","last_undegraded":"2023-07-14T19:27:05.795885+0000","last_fullsized":"2023-07-14T19:27:05.795885+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:18:46.077263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758100+0000","last_change":"2023-07-14T19:26:57.325889+0000","last_active":"2023-07-14T19:28:04.758100+0000","last_peered":"2023-07-14T19:28:04.758100+0000","last_clean":"2023-07-14T19:28:04.758100+0000","last_became_active":"2023-07-14T19:26:57.325580+0000","last_became_peered":"2023-07-14T19:26:57.325580+0000","last_unstale":"2023-07-14T19:28:04.758100+0000","last_undegraded":"2023-07-14T19:28:04.758100+0000","last_fullsized":"2023-07-14T19:28:04.758100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:40:30.860044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.790247+0000","last_change":"2023-07-14T19:26:56.496123+0000","last_active":"2023-07-14T19:27:05.790247+0000","last_peered":"2023-07-14T19:27:05.790247+0000","last_clean":"2023-07-14T19:27:05.790247+0000","last_became_active":"2023-07-14T19:26:56.495791+0000","last_became_peered":"2023-07-14T19:26:56.495791+0000","last_unstale":"2023-07-14T19:27:05.790247+0000","last_undegraded":"2023-07-14T19:27:05.790247+0000","last_fullsized":"2023-07-14T19:27:05.790247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:44:21.663771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758694+0000","last_change":"2023-07-14T19:26:57.330423+0000","last_active":"2023-07-14T19:28:04.758694+0000","last_peered":"2023-07-14T19:28:04.758694+0000","last_clean":"2023-07-14T19:28:04.758694+0000","last_became_active":"2023-07-14T19:26:57.330146+0000","last_became_peered":"2023-07-14T19:26:57.330146+0000","last_unstale":"2023-07-14T19:28:04.758694+0000","last_undegraded":"2023-07-14T19:28:04.758694+0000","last_fullsized":"2023-07-14T19:28:04.758694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:38:26.970479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952264+0000","last_change":"2023-07-14T19:26:56.498629+0000","last_active":"2023-07-14T19:27:05.952264+0000","last_peered":"2023-07-14T19:27:05.952264+0000","last_clean":"2023-07-14T19:27:05.952264+0000","last_became_active":"2023-07-14T19:26:56.498321+0000","last_became_peered":"2023-07-14T19:26:56.498321+0000","last_unstale":"2023-07-14T19:27:05.952264+0000","last_undegraded":"2023-07-14T19:27:05.952264+0000","last_fullsized":"2023-07-14T19:27:05.952264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:28.081451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769444+0000","last_change":"2023-07-14T19:26:57.328812+0000","last_active":"2023-07-14T19:28:04.769444+0000","last_peered":"2023-07-14T19:28:04.769444+0000","last_clean":"2023-07-14T19:28:04.769444+0000","last_became_active":"2023-07-14T19:26:57.328524+0000","last_became_peered":"2023-07-14T19:26:57.328524+0000","last_unstale":"2023-07-14T19:28:04.769444+0000","last_undegraded":"2023-07-14T19:28:04.769444+0000","last_fullsized":"2023-07-14T19:28:04.769444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:01:40.592640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.752609+0000","last_change":"2023-07-14T19:26:57.319058+0000","last_active":"2023-07-14T19:28:04.752609+0000","last_peered":"2023-07-14T19:28:04.752609+0000","last_clean":"2023-07-14T19:28:04.752609+0000","last_became_active":"2023-07-14T19:26:57.318584+0000","last_became_peered":"2023-07-14T19:26:57.318584+0000","last_unstale":"2023-07-14T19:28:04.752609+0000","last_undegraded":"2023-07-14T19:28:04.752609+0000","last_fullsized":"2023-07-14T19:28:04.752609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:37:29.089446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":103,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:20.654935+0000","last_change":"2023-07-14T19:26:56.498137+0000","last_active":"2023-07-14T19:28:20.654935+0000","last_peered":"2023-07-14T19:28:20.654935+0000","last_clean":"2023-07-14T19:28:20.654935+0000","last_became_active":"2023-07-14T19:26:56.497588+0000","last_became_peered":"2023-07-14T19:26:56.497588+0000","last_unstale":"2023-07-14T19:28:20.654935+0000","last_undegraded":"2023-07-14T19:28:20.654935+0000","last_fullsized":"2023-07-14T19:28:20.654935+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:52:38.820059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765118+0000","last_change":"2023-07-14T19:26:57.320080+0000","last_active":"2023-07-14T19:28:04.765118+0000","last_peered":"2023-07-14T19:28:04.765118+0000","last_clean":"2023-07-14T19:28:04.765118+0000","last_became_active":"2023-07-14T19:26:57.319776+0000","last_became_peered":"2023-07-14T19:26:57.319776+0000","last_unstale":"2023-07-14T19:28:04.765118+0000","last_undegraded":"2023-07-14T19:28:04.765118+0000","last_fullsized":"2023-07-14T19:28:04.765118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:58:20.144446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.789079+0000","last_change":"2023-07-14T19:26:56.497606+0000","last_active":"2023-07-14T19:27:05.789079+0000","last_peered":"2023-07-14T19:27:05.789079+0000","last_clean":"2023-07-14T19:27:05.789079+0000","last_became_active":"2023-07-14T19:26:56.497296+0000","last_became_peered":"2023-07-14T19:26:56.497296+0000","last_unstale":"2023-07-14T19:27:05.789079+0000","last_undegraded":"2023-07-14T19:27:05.789079+0000","last_fullsized":"2023-07-14T19:27:05.789079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:00:11.895409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952086+0000","last_change":"2023-07-14T19:25:58.253490+0000","last_active":"2023-07-14T19:27:05.952086+0000","last_peered":"2023-07-14T19:27:05.952086+0000","last_clean":"2023-07-14T19:27:05.952086+0000","last_became_active":"2023-07-14T19:25:58.253131+0000","last_became_peered":"2023-07-14T19:25:58.253131+0000","last_unstale":"2023-07-14T19:27:05.952086+0000","last_undegraded":"2023-07-14T19:27:05.952086+0000","last_fullsized":"2023-07-14T19:27:05.952086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:23:09.248219+0000","last_clean_scrub_stamp":"2023-07-14T19:23:09.248219+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:35:57.503183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756998+0000","last_change":"2023-07-14T19:26:57.321661+0000","last_active":"2023-07-14T19:28:04.756998+0000","last_peered":"2023-07-14T19:28:04.756998+0000","last_clean":"2023-07-14T19:28:04.756998+0000","last_became_active":"2023-07-14T19:26:57.321414+0000","last_became_peered":"2023-07-14T19:26:57.321414+0000","last_unstale":"2023-07-14T19:28:04.756998+0000","last_undegraded":"2023-07-14T19:28:04.756998+0000","last_fullsized":"2023-07-14T19:28:04.756998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:18:37.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952256+0000","last_change":"2023-07-14T19:26:56.498233+0000","last_active":"2023-07-14T19:27:05.952256+0000","last_peered":"2023-07-14T19:27:05.952256+0000","last_clean":"2023-07-14T19:27:05.952256+0000","last_became_active":"2023-07-14T19:26:56.497716+0000","last_became_peered":"2023-07-14T19:26:56.497716+0000","last_unstale":"2023-07-14T19:27:05.952256+0000","last_undegraded":"2023-07-14T19:27:05.952256+0000","last_fullsized":"2023-07-14T19:27:05.952256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T22:26:40.155552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795661+0000","last_change":"2023-07-14T19:26:56.496137+0000","last_active":"2023-07-14T19:27:05.795661+0000","last_peered":"2023-07-14T19:27:05.795661+0000","last_clean":"2023-07-14T19:27:05.795661+0000","last_became_active":"2023-07-14T19:26:56.495841+0000","last_became_peered":"2023-07-14T19:26:56.495841+0000","last_unstale":"2023-07-14T19:27:05.795661+0000","last_undegraded":"2023-07-14T19:27:05.795661+0000","last_fullsized":"2023-07-14T19:27:05.795661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:22:05.271093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760036+0000","last_change":"2023-07-14T19:26:57.321763+0000","last_active":"2023-07-14T19:28:04.760036+0000","last_peered":"2023-07-14T19:28:04.760036+0000","last_clean":"2023-07-14T19:28:04.760036+0000","last_became_active":"2023-07-14T19:26:57.321418+0000","last_became_peered":"2023-07-14T19:26:57.321418+0000","last_unstale":"2023-07-14T19:28:04.760036+0000","last_undegraded":"2023-07-14T19:28:04.760036+0000","last_fullsized":"2023-07-14T19:28:04.760036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:12:18.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'6","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:10.903427+0000","last_change":"2023-07-14T19:26:56.496639+0000","last_active":"2023-07-14T19:27:10.903427+0000","last_peered":"2023-07-14T19:27:10.903427+0000","last_clean":"2023-07-14T19:27:10.903427+0000","last_became_active":"2023-07-14T19:26:56.496321+0000","last_became_peered":"2023-07-14T19:26:56.496321+0000","last_unstale":"2023-07-14T19:27:10.903427+0000","last_undegraded":"2023-07-14T19:27:10.903427+0000","last_fullsized":"2023-07-14T19:27:10.903427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:32:26.769682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762774+0000","last_change":"2023-07-14T19:26:57.319506+0000","last_active":"2023-07-14T19:28:04.762774+0000","last_peered":"2023-07-14T19:28:04.762774+0000","last_clean":"2023-07-14T19:28:04.762774+0000","last_became_active":"2023-07-14T19:26:57.319161+0000","last_became_peered":"2023-07-14T19:26:57.319161+0000","last_unstale":"2023-07-14T19:28:04.762774+0000","last_undegraded":"2023-07-14T19:28:04.762774+0000","last_fullsized":"2023-07-14T19:28:04.762774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:03:14.066177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.788996+0000","last_change":"2023-07-14T19:26:56.499838+0000","last_active":"2023-07-14T19:27:05.788996+0000","last_peered":"2023-07-14T19:27:05.788996+0000","last_clean":"2023-07-14T19:27:05.788996+0000","last_became_active":"2023-07-14T19:26:56.499560+0000","last_became_peered":"2023-07-14T19:26:56.499560+0000","last_unstale":"2023-07-14T19:27:05.788996+0000","last_undegraded":"2023-07-14T19:27:05.788996+0000","last_fullsized":"2023-07-14T19:27:05.788996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:04:44.601003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.767489+0000","last_change":"2023-07-14T19:26:57.326238+0000","last_active":"2023-07-14T19:28:04.767489+0000","last_peered":"2023-07-14T19:28:04.767489+0000","last_clean":"2023-07-14T19:28:04.767489+0000","last_became_active":"2023-07-14T19:26:57.325926+0000","last_became_peered":"2023-07-14T19:26:57.325926+0000","last_unstale":"2023-07-14T19:28:04.767489+0000","last_undegraded":"2023-07-14T19:28:04.767489+0000","last_fullsized":"2023-07-14T19:28:04.767489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:45:38.640268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.903756+0000","last_change":"2023-07-14T19:26:56.504724+0000","last_active":"2023-07-14T19:27:05.903756+0000","last_peered":"2023-07-14T19:27:05.903756+0000","last_clean":"2023-07-14T19:27:05.903756+0000","last_became_active":"2023-07-14T19:26:56.504418+0000","last_became_peered":"2023-07-14T19:26:56.504418+0000","last_unstale":"2023-07-14T19:27:05.903756+0000","last_undegraded":"2023-07-14T19:27:05.903756+0000","last_fullsized":"2023-07-14T19:27:05.903756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:05:31.831826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.739189+0000","last_change":"2023-07-14T19:26:57.318763+0000","last_active":"2023-07-14T19:28:04.739189+0000","last_peered":"2023-07-14T19:28:04.739189+0000","last_clean":"2023-07-14T19:28:04.739189+0000","last_became_active":"2023-07-14T19:26:57.318434+0000","last_became_peered":"2023-07-14T19:26:57.318434+0000","last_unstale":"2023-07-14T19:28:04.739189+0000","last_undegraded":"2023-07-14T19:28:04.739189+0000","last_fullsized":"2023-07-14T19:28:04.739189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:17:50.824483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.787960+0000","last_change":"2023-07-14T19:26:56.502998+0000","last_active":"2023-07-14T19:27:05.787960+0000","last_peered":"2023-07-14T19:27:05.787960+0000","last_clean":"2023-07-14T19:27:05.787960+0000","last_became_active":"2023-07-14T19:26:56.501331+0000","last_became_peered":"2023-07-14T19:26:56.501331+0000","last_unstale":"2023-07-14T19:27:05.787960+0000","last_undegraded":"2023-07-14T19:27:05.787960+0000","last_fullsized":"2023-07-14T19:27:05.787960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T20:30:47.286623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764282+0000","last_change":"2023-07-14T19:26:57.321008+0000","last_active":"2023-07-14T19:28:04.764282+0000","last_peered":"2023-07-14T19:28:04.764282+0000","last_clean":"2023-07-14T19:28:04.764282+0000","last_became_active":"2023-07-14T19:26:57.320690+0000","last_became_peered":"2023-07-14T19:26:57.320690+0000","last_unstale":"2023-07-14T19:28:04.764282+0000","last_undegraded":"2023-07-14T19:28:04.764282+0000","last_fullsized":"2023-07-14T19:28:04.764282+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:21:20.246759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.806328+0000","last_change":"2023-07-14T19:27:05.805846+0000","last_active":"2023-07-14T19:27:05.806328+0000","last_peered":"2023-07-14T19:27:05.806328+0000","last_clean":"2023-07-14T19:27:05.806328+0000","last_became_active":"2023-07-14T19:27:05.796805+0000","last_became_peered":"2023-07-14T19:27:05.796805+0000","last_unstale":"2023-07-14T19:27:05.806328+0000","last_undegraded":"2023-07-14T19:27:05.806328+0000","last_fullsized":"2023-07-14T19:27:05.806328+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:47:58.264439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.771308+0000","last_change":"2023-07-14T19:26:57.328886+0000","last_active":"2023-07-14T19:28:04.771308+0000","last_peered":"2023-07-14T19:28:04.771308+0000","last_clean":"2023-07-14T19:28:04.771308+0000","last_became_active":"2023-07-14T19:26:57.328571+0000","last_became_peered":"2023-07-14T19:26:57.328571+0000","last_unstale":"2023-07-14T19:28:04.771308+0000","last_undegraded":"2023-07-14T19:28:04.771308+0000","last_fullsized":"2023-07-14T19:28:04.771308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T23:54:50.760237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795490+0000","last_change":"2023-07-14T19:26:56.496560+0000","last_active":"2023-07-14T19:27:05.795490+0000","last_peered":"2023-07-14T19:27:05.795490+0000","last_clean":"2023-07-14T19:27:05.795490+0000","last_became_active":"2023-07-14T19:26:56.494053+0000","last_became_peered":"2023-07-14T19:26:56.494053+0000","last_unstale":"2023-07-14T19:27:05.795490+0000","last_undegraded":"2023-07-14T19:27:05.795490+0000","last_fullsized":"2023-07-14T19:27:05.795490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:10:28.450015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760928+0000","last_change":"2023-07-14T19:26:57.321457+0000","last_active":"2023-07-14T19:28:04.760928+0000","last_peered":"2023-07-14T19:28:04.760928+0000","last_clean":"2023-07-14T19:28:04.760928+0000","last_became_active":"2023-07-14T19:26:57.321089+0000","last_became_peered":"2023-07-14T19:26:57.321089+0000","last_unstale":"2023-07-14T19:28:04.760928+0000","last_undegraded":"2023-07-14T19:28:04.760928+0000","last_fullsized":"2023-07-14T19:28:04.760928+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:47:05.779967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.795623+0000","last_change":"2023-07-14T19:26:56.497958+0000","last_active":"2023-07-14T19:27:05.795623+0000","last_peered":"2023-07-14T19:27:05.795623+0000","last_clean":"2023-07-14T19:27:05.795623+0000","last_became_active":"2023-07-14T19:26:56.496949+0000","last_became_peered":"2023-07-14T19:26:56.496949+0000","last_unstale":"2023-07-14T19:27:05.795623+0000","last_undegraded":"2023-07-14T19:27:05.795623+0000","last_fullsized":"2023-07-14T19:27:05.795623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:42.189785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.766892+0000","last_change":"2023-07-14T19:26:57.326194+0000","last_active":"2023-07-14T19:28:04.766892+0000","last_peered":"2023-07-14T19:28:04.766892+0000","last_clean":"2023-07-14T19:28:04.766892+0000","last_became_active":"2023-07-14T19:26:57.325910+0000","last_became_peered":"2023-07-14T19:26:57.325910+0000","last_unstale":"2023-07-14T19:28:04.766892+0000","last_undegraded":"2023-07-14T19:28:04.766892+0000","last_fullsized":"2023-07-14T19:28:04.766892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T03:31:52.615160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.952458+0000","last_change":"2023-07-14T19:26:56.502731+0000","last_active":"2023-07-14T19:27:05.952458+0000","last_peered":"2023-07-14T19:27:05.952458+0000","last_clean":"2023-07-14T19:27:05.952458+0000","last_became_active":"2023-07-14T19:26:56.502433+0000","last_became_peered":"2023-07-14T19:26:56.502433+0000","last_unstale":"2023-07-14T19:27:05.952458+0000","last_undegraded":"2023-07-14T19:27:05.952458+0000","last_fullsized":"2023-07-14T19:27:05.952458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:53:34.349607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764119+0000","last_change":"2023-07-14T19:26:57.323889+0000","last_active":"2023-07-14T19:28:04.764119+0000","last_peered":"2023-07-14T19:28:04.764119+0000","last_clean":"2023-07-14T19:28:04.764119+0000","last_became_active":"2023-07-14T19:26:57.323557+0000","last_became_peered":"2023-07-14T19:26:57.323557+0000","last_unstale":"2023-07-14T19:28:04.764119+0000","last_undegraded":"2023-07-14T19:28:04.764119+0000","last_fullsized":"2023-07-14T19:28:04.764119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:50:19.375696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:27:05.951776+0000","last_change":"2023-07-14T19:26:56.503997+0000","last_active":"2023-07-14T19:27:05.951776+0000","last_peered":"2023-07-14T19:27:05.951776+0000","last_clean":"2023-07-14T19:27:05.951776+0000","last_became_active":"2023-07-14T19:26:56.503706+0000","last_became_peered":"2023-07-14T19:26:56.503706+0000","last_unstale":"2023-07-14T19:27:05.951776+0000","last_undegraded":"2023-07-14T19:27:05.951776+0000","last_fullsized":"2023-07-14T19:27:05.951776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:55.477070+0000","last_clean_scrub_stamp":"2023-07-14T19:26:55.477070+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:04:55.101835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.758103+0000","last_change":"2023-07-14T19:26:57.326523+0000","last_active":"2023-07-14T19:28:04.758103+0000","last_peered":"2023-07-14T19:28:04.758103+0000","last_clean":"2023-07-14T19:28:04.758103+0000","last_became_active":"2023-07-14T19:26:57.326209+0000","last_became_peered":"2023-07-14T19:26:57.326209+0000","last_unstale":"2023-07-14T19:28:04.758103+0000","last_undegraded":"2023-07-14T19:28:04.758103+0000","last_fullsized":"2023-07-14T19:28:04.758103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:56:02.535273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.762361+0000","last_change":"2023-07-14T19:26:57.324909+0000","last_active":"2023-07-14T19:28:04.762361+0000","last_peered":"2023-07-14T19:28:04.762361+0000","last_clean":"2023-07-14T19:28:04.762361+0000","last_became_active":"2023-07-14T19:26:57.324578+0000","last_became_peered":"2023-07-14T19:26:57.324578+0000","last_unstale":"2023-07-14T19:28:04.762361+0000","last_undegraded":"2023-07-14T19:28:04.762361+0000","last_fullsized":"2023-07-14T19:28:04.762361+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:03:59.337396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759168+0000","last_change":"2023-07-14T19:26:57.327245+0000","last_active":"2023-07-14T19:28:04.759168+0000","last_peered":"2023-07-14T19:28:04.759168+0000","last_clean":"2023-07-14T19:28:04.759168+0000","last_became_active":"2023-07-14T19:26:57.326950+0000","last_became_peered":"2023-07-14T19:26:57.326950+0000","last_unstale":"2023-07-14T19:28:04.759168+0000","last_undegraded":"2023-07-14T19:28:04.759168+0000","last_fullsized":"2023-07-14T19:28:04.759168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:26:45.117785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.763076+0000","last_change":"2023-07-14T19:26:57.320527+0000","last_active":"2023-07-14T19:28:04.763076+0000","last_peered":"2023-07-14T19:28:04.763076+0000","last_clean":"2023-07-14T19:28:04.763076+0000","last_became_active":"2023-07-14T19:26:57.320253+0000","last_became_peered":"2023-07-14T19:26:57.320253+0000","last_unstale":"2023-07-14T19:28:04.763076+0000","last_undegraded":"2023-07-14T19:28:04.763076+0000","last_fullsized":"2023-07-14T19:28:04.763076+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T19:59:59.431251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.773029+0000","last_change":"2023-07-14T19:26:57.323788+0000","last_active":"2023-07-14T19:28:04.773029+0000","last_peered":"2023-07-14T19:28:04.773029+0000","last_clean":"2023-07-14T19:28:04.773029+0000","last_became_active":"2023-07-14T19:26:57.323382+0000","last_became_peered":"2023-07-14T19:26:57.323382+0000","last_unstale":"2023-07-14T19:28:04.773029+0000","last_undegraded":"2023-07-14T19:28:04.773029+0000","last_fullsized":"2023-07-14T19:28:04.773029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:40:42.135050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.764144+0000","last_change":"2023-07-14T19:26:57.319715+0000","last_active":"2023-07-14T19:28:04.764144+0000","last_peered":"2023-07-14T19:28:04.764144+0000","last_clean":"2023-07-14T19:28:04.764144+0000","last_became_active":"2023-07-14T19:26:57.319379+0000","last_became_peered":"2023-07-14T19:26:57.319379+0000","last_unstale":"2023-07-14T19:28:04.764144+0000","last_undegraded":"2023-07-14T19:28:04.764144+0000","last_fullsized":"2023-07-14T19:28:04.764144+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:02:24.075259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.760086+0000","last_change":"2023-07-14T19:26:57.317554+0000","last_active":"2023-07-14T19:28:04.760086+0000","last_peered":"2023-07-14T19:28:04.760086+0000","last_clean":"2023-07-14T19:28:04.760086+0000","last_became_active":"2023-07-14T19:26:57.317216+0000","last_became_peered":"2023-07-14T19:26:57.317216+0000","last_unstale":"2023-07-14T19:28:04.760086+0000","last_undegraded":"2023-07-14T19:28:04.760086+0000","last_fullsized":"2023-07-14T19:28:04.760086+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T02:28:50.553918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.750630+0000","last_change":"2023-07-14T19:26:57.323006+0000","last_active":"2023-07-14T19:28:04.750630+0000","last_peered":"2023-07-14T19:28:04.750630+0000","last_clean":"2023-07-14T19:28:04.750630+0000","last_became_active":"2023-07-14T19:26:57.322724+0000","last_became_peered":"2023-07-14T19:26:57.322724+0000","last_unstale":"2023-07-14T19:28:04.750630+0000","last_undegraded":"2023-07-14T19:28:04.750630+0000","last_fullsized":"2023-07-14T19:28:04.750630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:43:45.456973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.756375+0000","last_change":"2023-07-14T19:26:57.326294+0000","last_active":"2023-07-14T19:28:04.756375+0000","last_peered":"2023-07-14T19:28:04.756375+0000","last_clean":"2023-07-14T19:28:04.756375+0000","last_became_active":"2023-07-14T19:26:57.325931+0000","last_became_peered":"2023-07-14T19:26:57.325931+0000","last_unstale":"2023-07-14T19:28:04.756375+0000","last_undegraded":"2023-07-14T19:28:04.756375+0000","last_fullsized":"2023-07-14T19:28:04.756375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T00:35:43.308371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.761507+0000","last_change":"2023-07-14T19:26:57.311118+0000","last_active":"2023-07-14T19:28:04.761507+0000","last_peered":"2023-07-14T19:28:04.761507+0000","last_clean":"2023-07-14T19:28:04.761507+0000","last_became_active":"2023-07-14T19:26:57.310833+0000","last_became_peered":"2023-07-14T19:26:57.310833+0000","last_unstale":"2023-07-14T19:28:04.761507+0000","last_undegraded":"2023-07-14T19:28:04.761507+0000","last_fullsized":"2023-07-14T19:28:04.761507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T07:02:45.526660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759622+0000","last_change":"2023-07-14T19:26:57.321675+0000","last_active":"2023-07-14T19:28:04.759622+0000","last_peered":"2023-07-14T19:28:04.759622+0000","last_clean":"2023-07-14T19:28:04.759622+0000","last_became_active":"2023-07-14T19:26:57.321372+0000","last_became_peered":"2023-07-14T19:26:57.321372+0000","last_unstale":"2023-07-14T19:28:04.759622+0000","last_undegraded":"2023-07-14T19:28:04.759622+0000","last_fullsized":"2023-07-14T19:28:04.759622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:14:50.377751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.765248+0000","last_change":"2023-07-14T19:26:57.325347+0000","last_active":"2023-07-14T19:28:04.765248+0000","last_peered":"2023-07-14T19:28:04.765248+0000","last_clean":"2023-07-14T19:28:04.765248+0000","last_became_active":"2023-07-14T19:26:57.324856+0000","last_became_peered":"2023-07-14T19:26:57.324856+0000","last_unstale":"2023-07-14T19:28:04.765248+0000","last_undegraded":"2023-07-14T19:28:04.765248+0000","last_fullsized":"2023-07-14T19:28:04.765248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T05:51:09.749154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.769059+0000","last_change":"2023-07-14T19:26:57.320602+0000","last_active":"2023-07-14T19:28:04.769059+0000","last_peered":"2023-07-14T19:28:04.769059+0000","last_clean":"2023-07-14T19:28:04.769059+0000","last_became_active":"2023-07-14T19:26:57.320272+0000","last_became_peered":"2023-07-14T19:26:57.320272+0000","last_unstale":"2023-07-14T19:28:04.769059+0000","last_undegraded":"2023-07-14T19:28:04.769059+0000","last_fullsized":"2023-07-14T19:28:04.769059+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T21:07:02.143813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-14T19:28:04.759931+0000","last_change":"2023-07-14T19:27:05.796530+0000","last_active":"2023-07-14T19:28:04.759931+0000","last_peered":"2023-07-14T19:28:04.759931+0000","last_clean":"2023-07-14T19:28:04.759931+0000","last_became_active":"2023-07-14T19:27:05.796148+0000","last_became_peered":"2023-07-14T19:27:05.796148+0000","last_unstale":"2023-07-14T19:28:04.759931+0000","last_undegraded":"2023-07-14T19:28:04.759931+0000","last_fullsized":"2023-07-14T19:28:04.759931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-14T19:26:56.478920+0000","last_clean_scrub_stamp":"2023-07-14T19:26:56.478920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-16T06:30:52.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212784,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61252,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":505,"num_read_kb":318,"num_write":157,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":192078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514525,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":431808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.58799999999999997,"15min":0.60199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.69499999999999995,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.499},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.86499999999999999,"15min":0.92100000000000004},"min":{"1min":0.39000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":5.8380000000000001,"15min":5.8380000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.79300000000000004,"5min":0.874,"15min":0.874},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77500000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Jul 14 19:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.63},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66400000000000003,"15min":0.65300000000000002},"min":{"1min":0.54300000000000004,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.53000000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.56499999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.87,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.51600000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.65000000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.88700000000000001,"5min":1.179,"15min":1.179},"last":0.86699999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.90500000000000003}]},{"osd":9,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89900000000000002,"15min":0.91300000000000003},"min":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.96599999999999997,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.98199999999999998,"15min":1.0149999999999999},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1060000000000001,"5min":5.0170000000000003,"15min":5.0170000000000003},"last":0.89400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:28:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.998,"5min":1.083,"15min":1.083},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.82799999999999996,"15min":0.82299999999999995},"min":{"1min":0.66200000000000003,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.016,"5min":1.375,"15min":1.375},"last":0.88}]}]},{"osd":10,"up_from":69,"seq":296352743457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":1080420,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.79100000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42199999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.47099999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.69799999999999995,"15min":0.72299999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80400000000000005,"5min":0.998,"15min":0.998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.66400000000000003,"15min":0.68400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70299999999999996,"5min":1.222,"15min":1.222},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.63,"15min":0.64200000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.67000000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78200000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.879,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.5,"15min":0.5},"max":{"1min":0.89700000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.81799999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.76800000000000002,"15min":0.78800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82399999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.76800000000000002,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83599999999999997,"5min":1.24,"15min":1.24},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79700000000000004,"15min":0.80500000000000005},"min":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.88200000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.874},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.86699999999999999},"min":{"1min":0.64000000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95799999999999996,"5min":1.095,"15min":1.095},"last":0.72899999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.63100000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.82399999999999995,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.88100000000000001},"min":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93000000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Jul 14 19:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83599999999999997},"min":{"1min":0.61499999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.93100000000000005,"5min":1.02,"15min":1.02},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79500000000000004,"15min":0.80100000000000005},"min":{"1min":0.627,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.94599999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.89200000000000002}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":432010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.42999999999999999,"15min":0.441},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57499999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.48699999999999999,"15min":0.50600000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.55600000000000005,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.438}]},{"osd":1,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79000000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77900000000000003,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.70699999999999996,"15min":0.73499999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72199999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.58799999999999997,"15min":0.59999999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67700000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67100000000000004,"5min":0.753,"15min":0.753},"last":0.502},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.53600000000000003,"15min":0.55600000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76300000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.36199999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.64200000000000002,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.74199999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.67000000000000004,"15min":0.68200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.85299999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.66100000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80000000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.59599999999999997,"15min":0.61199999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.66500000000000004,"15min":0.68799999999999994},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77300000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.89200000000000002,"5min":1.081,"15min":1.081},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89300000000000002,"5min":1.087,"15min":1.087},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.74299999999999999,"15min":0.76200000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81999999999999995,"5min":1.141,"15min":1.141},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83499999999999996,"5min":1.367,"15min":1.367},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 14 19:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.52300000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.90300000000000002,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.73999999999999999,"15min":0.752},"min":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.86599999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.70399999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":431930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64000000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46999999999999997,"15min":0.47499999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.5,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.504}]},{"osd":3,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.751,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88700000000000001,"5min":1.534,"15min":1.534},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.628,"15min":0.628},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86599999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.95599999999999996}]},{"osd":5,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.311,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.74099999999999999,"15min":0.754},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1120000000000001,"5min":1.675,"15min":1.675},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Jul 14 19:27:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.93500000000000005}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.73699999999999999},"min":{"1min":0.56699999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.627,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.75,"15min":0.72599999999999998},"min":{"1min":0.72499999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0189999999999999,"5min":1.03,"15min":1.03},"last":0.99299999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83299999999999996}]}]},{"osd":7,"up_from":50,"seq":214748364845,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":1023579,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.312,"15min":0.312},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.59499999999999997,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":5.8819999999999997,"5min":5.8819999999999997,"15min":5.8819999999999997},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.33000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.61799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.34999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.5,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.502,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84799999999999998,"5min":1.024,"15min":1.024},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.57899999999999996},"min":{"1min":0.47399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82599999999999996,"5min":4.8019999999999996,"15min":4.8019999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.52000000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92500000000000004,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.745,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.52600000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.64400000000000002},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97499999999999998,"5min":1.127,"15min":1.127},"last":0.86399999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.55800000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.91400000000000003,"5min":1.954,"15min":1.954},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.73699999999999999,"15min":0.70599999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.878,"5min":5.8310000000000004,"15min":5.8310000000000004},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93000000000000005,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.503,"5min":0.498,"15min":0.498},"max":{"1min":0.86499999999999999,"5min":1,"15min":1},"last":0.95299999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.76900000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":4.7610000000000001,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.625,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.93899999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.57799999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94599999999999995,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.91100000000000003}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.98899999999999999}]}]},{"osd":6,"up_from":45,"seq":193273528369,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":431405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.40200000000000002,"15min":0.41399999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.374},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.54200000000000004,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.376,"5min":0.373,"15min":0.373},"max":{"1min":0.68899999999999995,"5min":1.212,"15min":1.212},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.63800000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.621}]},{"osd":2,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57399999999999995,"15min":0.58399999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.46600000000000003,"15min":0.47199999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.625,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.48899999999999999,"15min":0.497},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.56100000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60999999999999999,"15min":0.61599999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75600000000000001,"5min":1.115,"15min":1.115},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61799999999999999,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85099999999999998,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":1.032,"15min":1.032},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70399999999999996,"15min":0.72199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.077,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.67100000000000004,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93100000000000005,"5min":1.151,"15min":1.151},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:28:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.68500000000000005,"15min":0.72199999999999998},"min":{"1min":0.433,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.68200000000000005,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.68300000000000005,"15min":0.70699999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.68799999999999994,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.59999999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.69699999999999995,"15min":0.71999999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.3480000000000001,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.72299999999999998,"15min":0.754},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67600000000000005,"5min":1.55,"15min":1.55},"last":0.65000000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:28:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.746,"15min":0.76400000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.67600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":492401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.36199999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76900000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.57799999999999996,"15min":0.50700000000000001},"min":{"1min":0.42999999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.876},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.57199999999999995,"15min":0.51200000000000001},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.84699999999999998,"5min":1.302,"15min":1.302},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.46200000000000002,"15min":0.41899999999999998},"min":{"1min":0.41199999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.626},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.48999999999999999,"15min":0.45600000000000002},"min":{"1min":0.38100000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74399999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63100000000000001,"15min":0.58499999999999996},"min":{"1min":0.51200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.627,"15min":0.57899999999999996},"min":{"1min":0.53300000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.52400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71899999999999997,"5min":1.042,"15min":1.042},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.51000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Jul 14 19:27:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.60699999999999998,"15min":0.57499999999999996},"min":{"1min":0.44700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.879,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.65800000000000003,"15min":0.626},"min":{"1min":0.58699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.98499999999999999}]},{"osd":8,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.69199999999999995,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.67900000000000005,"15min":0.63900000000000001},"min":{"1min":0.60599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84599999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92700000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.72899999999999998,"15min":0.71599999999999997},"min":{"1min":0.66000000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.96399999999999997,"5min":1.004,"15min":1.004},"last":0.93600000000000005}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.52400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94899999999999995,"5min":1.087,"15min":1.087},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.65700000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.996}]},{"osd":11,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":1.042},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.0509999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":432670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.48699999999999999},"min":{"1min":0.34999999999999998,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.874,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.36699999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.749,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.55400000000000005},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71199999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.40400000000000003,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.376},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.55500000000000005,"15min":0.55700000000000005},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.64200000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.435}]},{"osd":7,"last update":"Fri Jul 14 19:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63600000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71199999999999997}]},{"osd":9,"last update":"Fri Jul 14 19:27:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.628,"15min":0.64400000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76900000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.63400000000000001,"15min":0.64600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.78700000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.77300000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74399999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:28:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64900000000000002,"15min":0.65600000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.71199999999999997,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.77600000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":494200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.47599999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.55200000000000005,"15min":0.48699999999999999},"min":{"1min":0.40799999999999997,"5min":0.316,"15min":0.316},"max":{"1min":3.8759999999999999,"5min":3.8759999999999999,"15min":3.8759999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57499999999999996,"15min":0.51800000000000002},"min":{"1min":0.40699999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58699999999999997,"15min":0.52400000000000002},"min":{"1min":0.375,"5min":0.255,"15min":0.255},"max":{"1min":0.83899999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.60799999999999998,"15min":0.56499999999999995},"min":{"1min":0.40600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":4.9180000000000001,"5min":4.9180000000000001,"15min":4.9180000000000001},"last":0.436},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":4.9359999999999999,"5min":4.9359999999999999,"15min":4.9359999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.45100000000000001},"min":{"1min":0.34499999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.75600000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.56100000000000005,"15min":0.52500000000000002},"min":{"1min":0.52400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.78000000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.65400000000000003,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.7560000000000002,"5min":4.7560000000000002,"15min":4.7560000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jul 14 19:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.626}]},{"osd":8,"last update":"Fri Jul 14 19:28:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91900000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.38800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.88200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 14 19:27:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.70299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.8039999999999998,"5min":2.8039999999999998,"15min":2.8039999999999998},"last":0.753},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.69699999999999995}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.71699999999999997,"15min":0.69799999999999995},"min":{"1min":0.65400000000000003,"5min":0.505,"15min":0.505},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.754,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.85799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149757,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":431722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.32100000000000001},"min":{"1min":0.28299999999999997,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"min":{"1min":0.27300000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.55300000000000005,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68000000000000005,"5min":1.202,"15min":1.202},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.503},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.44400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71999999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58399999999999996,"15min":0.56899999999999995},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.627},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.46100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.75700000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 14 19:28:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.35299999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89200000000000002,"5min":1.167,"15min":1.167},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Jul 14 19:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.54300000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.442},{"interface":"front","average":{"1min":0.442,"5min":0.46400000000000002,"15min":0.45600000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61799999999999999,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.373}]},{"osd":7,"last update":"Fri Jul 14 19:27:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.57799999999999996},"min":{"1min":0.441,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68200000000000005,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72399999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.63700000000000001}]},{"osd":8,"last update":"Fri Jul 14 19:26:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.65700000000000003,"15min":0.67100000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78800000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51400000000000001}]},{"osd":10,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.71199999999999997,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.77100000000000002,"5min":1.006,"15min":1.006},"last":0.66500000000000004}]},{"osd":11,"last update":"Fri Jul 14 19:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67500000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":432106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51800000000000002,"15min":0.5},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69599999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.39300000000000002,"15min":0.372},"min":{"1min":0.32200000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.313,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.81000000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60299999999999998,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39100000000000001}]},{"osd":3,"last update":"Fri Jul 14 19:27:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.872,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.499,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92600000000000005,"5min":1.036,"15min":1.036},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55800000000000005,"15min":0.52800000000000002},"min":{"1min":0.44700000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.57499999999999996,"15min":0.55200000000000005},"min":{"1min":0.442,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.41399999999999998}]},{"osd":8,"last update":"Fri Jul 14 19:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58199999999999996}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85899999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72399999999999998,"15min":0.71799999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64400000000000002}]},{"osd":10,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.56200000000000006,"5min":0.435,"15min":0.435},"max":{"1min":0.91000000000000003,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.91000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61099999999999999}]},{"osd":11,"last update":"Fri Jul 14 19:27:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":1023311,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.68200000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.40999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.996,"15min":0.996},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.496,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76600000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83499999999999996,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.44600000000000001,"15min":0.42799999999999999},"min":{"1min":0.35799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.61799999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.75,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":1.022,"15min":1.022},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.45300000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77100000000000002,"5min":1.044,"15min":1.044},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.64700000000000002,"15min":0.628},"min":{"1min":0.58799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.624,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.91700000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.47199999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.81299999999999994,"5min":1.087,"15min":1.087},"last":0.749},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.56000000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":1.113,"15min":1.113},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Jul 14 19:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.67400000000000004,"15min":0.65500000000000003},"min":{"1min":0.52000000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95299999999999996,"5min":1.004,"15min":1.004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.59399999999999997,"5min":0.377,"15min":0.377},"max":{"1min":0.91400000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri Jul 14 19:28:19 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65700000000000003,"15min":0.63100000000000001},"min":{"1min":0.46700000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93799999999999994,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.57899999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.875}]},{"osd":9,"last update":"Fri Jul 14 19:27:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72099999999999997,"15min":0.71099999999999997},"min":{"1min":0.621,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Jul 14 19:28:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.76500000000000001,"15min":0.748},"min":{"1min":0.73699999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.753,"15min":0.747},"min":{"1min":0.64600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":0.88400000000000001,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34099,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35737,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-14T19:28:24.161 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:24.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:23 smithi133 ceph-mon[128894]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T19:28:24.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:23 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1643966330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:24.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:23 smithi161 ceph-mon[129184]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-07-14T19:28:24.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:23 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1643966330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:24.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 -- 172.21.15.82:0/1309298770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 msgr2=0x7f6d1c0fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 --2- 172.21.15.82:0/1309298770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c0fe8d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009990 tx=0x7f6d14009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 -- 172.21.15.82:0/1309298770 shutdown_connections 2023-07-14T19:28:24.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 --2- 172.21.15.82:0/1309298770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d1c0ff7c0 0x7f6d1c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 --2- 172.21.15.82:0/1309298770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c0fe8d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 --2- 172.21.15.82:0/1309298770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 0x7f6d1c0ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 -- 172.21.15.82:0/1309298770 >> 172.21.15.82:0/1309298770 conn(0x7f6d1c0fa150 msgr2=0x7f6d1c0fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:24.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 -- 172.21.15.82:0/1309298770 shutdown_connections 2023-07-14T19:28:24.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.312+0000 7f6d21813700 1 -- 172.21.15.82:0/1309298770 wait complete. 2023-07-14T19:28:24.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 Processor -- start 2023-07-14T19:28:24.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 -- start start 2023-07-14T19:28:24.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 0x7f6d1c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d1c0ff7c0 0x7f6d1c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19f820 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19f990 con 0x7f6d1c0fee10 2023-07-14T19:28:24.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d21813700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c19fb50 con 0x7f6d1c0ff7c0 2023-07-14T19:28:24.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42214/0 (socket says 172.21.15.82:42214) 2023-07-14T19:28:24.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d1affd700 1 -- 172.21.15.82:0/3101396024 learned_addr learned my addr 172.21.15.82:0/3101396024 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:24.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.313+0000 7f6d1b7fe700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d1c0ff7c0 0x7f6d1c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1a7fc700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 0x7f6d1c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1affd700 1 -- 172.21.15.82:0/3101396024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d1c0ff7c0 msgr2=0x7f6d1c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1affd700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d1c0ff7c0 0x7f6d1c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1affd700 1 -- 172.21.15.82:0/3101396024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 msgr2=0x7f6d1c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1affd700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 0x7f6d1c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1affd700 1 -- 172.21.15.82:0/3101396024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140095f0 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1a7fc700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 0x7f6d1c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:24.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d1affd700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c197cb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d140059d0 tx=0x7f6d140047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:24.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1401d070 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c19fdd0 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d1400fc40 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14017650 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.314+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c1a01a0 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.328 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.315+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c109300 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.329 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.316+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6d14022470 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.316+0000 7f6d20811700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6d080a11a0 0x7f6d080a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.330 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.317+0000 7f6d1a7fc700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6d080a11a0 0x7f6d080a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.317+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f6d140cdf20 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.331 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.317+0000 7f6d1a7fc700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6d080a11a0 0x7f6d080a3650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c1991b0 tx=0x7f6d04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:24.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.321+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f6d14097720 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.515+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c0612f0 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.516+0000 7f6d20811700 1 -- 172.21.15.82:0/3101396024 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f6d1409ad40 con 0x7f6d1c0fe4c0 2023-07-14T19:28:24.518 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:24.518 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:24.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6d080a11a0 msgr2=0x7f6d080a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6d080a11a0 0x7f6d080a3650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c1991b0 tx=0x7f6d04009260 comp rx=0 tx=0).stop 2023-07-14T19:28:24.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 msgr2=0x7f6d1c197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c197cb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d140059d0 tx=0x7f6d140047a0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.520 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 shutdown_connections 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d1c0ff7c0 0x7f6d1c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f6d080a11a0 0x7f6d080a3650 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d1c0fe4c0 0x7f6d1c197cb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 --2- 172.21.15.82:0/3101396024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d1c0fee10 0x7f6d1c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 >> 172.21.15.82:0/3101396024 conn(0x7f6d1c0fa150 msgr2=0x7f6d1c0fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 shutdown_connections 2023-07-14T19:28:24.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.518+0000 7f6d21813700 1 -- 172.21.15.82:0/3101396024 wait complete. 2023-07-14T19:28:24.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:28:24.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:28:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 -- 172.21.15.82:0/791046835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0fed20 msgr2=0x7fca9c0ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/791046835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0fed20 0x7fca9c0ff190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fca90009990 tx=0x7fca90009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 -- 172.21.15.82:0/791046835 shutdown_connections 2023-07-14T19:28:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/791046835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/791046835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca9c1000d0 0x7fca9c1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/791046835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0fed20 0x7fca9c0ff190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 -- 172.21.15.82:0/791046835 >> 172.21.15.82:0/791046835 conn(0x7fca9c0f98a0 msgr2=0x7fca9c0fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 -- 172.21.15.82:0/791046835 shutdown_connections 2023-07-14T19:28:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.655+0000 7fcaa29b2700 1 -- 172.21.15.82:0/791046835 wait complete. 2023-07-14T19:28:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 Processor -- start 2023-07-14T19:28:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 -- start start 2023-07-14T19:28:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c0fed20 0x7fca9c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca9c1000d0 0x7fca9c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c19d510 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c19d650 con 0x7fca9c1000d0 2023-07-14T19:28:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fcaa29b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c19d7e0 con 0x7fca9c0fed20 2023-07-14T19:28:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fca9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fca9b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42238/0 (socket says 172.21.15.82:42238) 2023-07-14T19:28:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.656+0000 7fca9b7fe700 1 -- 172.21.15.82:0/2357731656 learned_addr learned my addr 172.21.15.82:0/2357731656 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9bfff700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c0fed20 0x7fca9c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fcaa0f4f700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca9c1000d0 0x7fca9c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9b7fe700 1 -- 172.21.15.82:0/2357731656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c0fed20 msgr2=0x7fca9c195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9b7fe700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c0fed20 0x7fca9c195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9b7fe700 1 -- 172.21.15.82:0/2357731656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca9c1000d0 msgr2=0x7fca9c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9b7fe700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca9c1000d0 0x7fca9c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9b7fe700 1 -- 172.21.15.82:0/2357731656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900095f0 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca9b7fe700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c195ed0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fca900095c0 tx=0x7fca9000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9001d070 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca90005030 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c19da60 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90004ab0 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.657+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c19de80 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.658+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c10a310 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.660+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fca90004c50 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.660+0000 7fca997fa700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fca840a1150 0x7fca840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.661+0000 7fca9bfff700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fca840a1150 0x7fca840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.661+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fca900cdb90 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.661+0000 7fca9bfff700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fca840a1150 0x7fca840a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca9c067270 tx=0x7fca8c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:24.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.664+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fca90097430 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.854 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.853+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca9c19e490 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.855 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.853+0000 7fca997fa700 1 -- 172.21.15.82:0/2357731656 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca9009aa50 con 0x7fca9c0ff6d0 2023-07-14T19:28:24.855 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:28:24.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fca840a1150 msgr2=0x7fca840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fca840a1150 0x7fca840a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca9c067270 tx=0x7fca8c00b410 comp rx=0 tx=0).stop 2023-07-14T19:28:24.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 msgr2=0x7fca9c195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.857 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c195ed0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fca900095c0 tx=0x7fca9000f690 comp rx=0 tx=0).stop 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 shutdown_connections 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca9c1000d0 0x7fca9c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c0fed20 0x7fca9c195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fca840a1150 0x7fca840a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 --2- 172.21.15.82:0/2357731656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca9c0ff6d0 0x7fca9c195ed0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 >> 172.21.15.82:0/2357731656 conn(0x7fca9c0f98a0 msgr2=0x7fca9c0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 shutdown_connections 2023-07-14T19:28:24.858 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.855+0000 7fcaa29b2700 1 -- 172.21.15.82:0/2357731656 wait complete. 2023-07-14T19:28:24.869 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:28:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 -- 172.21.15.82:0/419755904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 msgr2=0x7f30d4100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 --2- 172.21.15.82:0/419755904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4100150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009990 tx=0x7f30d0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 -- 172.21.15.82:0/419755904 shutdown_connections 2023-07-14T19:28:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 --2- 172.21.15.82:0/419755904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4100690 0x7f30d4106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 --2- 172.21.15.82:0/419755904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30d40feb10 0x7f30d40fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.994 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 --2- 172.21.15.82:0/419755904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4100150 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 -- 172.21.15.82:0/419755904 >> 172.21.15.82:0/419755904 conn(0x7f30d40fa130 msgr2=0x7f30d40fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:24.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 -- 172.21.15.82:0/419755904 shutdown_connections 2023-07-14T19:28:24.995 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.992+0000 7f30dba8c700 1 -- 172.21.15.82:0/419755904 wait complete. 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 Processor -- start 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 -- start start 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30d40feb10 0x7f30d41948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4100690 0x7f30d419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d419c460 con 0x7f30d40ffd00 2023-07-14T19:28:24.997 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d419c5a0 con 0x7f30d4100690 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30dba8c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d419c730 con 0x7f30d40feb10 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30d9027700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30d9027700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4194e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42250/0 (socket says 172.21.15.82:42250) 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30d9027700 1 -- 172.21.15.82:0/611473500 learned_addr learned my addr 172.21.15.82:0/611473500 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30d9828700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30d40feb10 0x7f30d41948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.993+0000 7f30da029700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4100690 0x7f30d419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:24.998 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30d9027700 1 -- 172.21.15.82:0/611473500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30d40feb10 msgr2=0x7f30d41948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30d9027700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30d40feb10 0x7f30d41948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30d9027700 1 -- 172.21.15.82:0/611473500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4100690 msgr2=0x7f30d419be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30d9027700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4100690 0x7f30d419be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30d9027700 1 -- 172.21.15.82:0/611473500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d00095f0 con 0x7f30d40ffd00 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30d9027700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4194e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f30d0004760 tx=0x7f30d0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d001d070 con 0x7f30d40ffd00 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30d0004cf0 con 0x7f30d40ffd00 2023-07-14T19:28:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d419c9b0 con 0x7f30d40ffd00 2023-07-14T19:28:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0003d00 con 0x7f30d40ffd00 2023-07-14T19:28:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.994+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d419ce50 con 0x7f30d40ffd00 2023-07-14T19:28:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.995+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d4106150 con 0x7f30d40ffd00 2023-07-14T19:28:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.996+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f30d0005000 con 0x7f30d40ffd00 2023-07-14T19:28:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.997+0000 7f30caffd700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30c00a11d0 0x7f30c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.997+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f30d00cdcc0 con 0x7f30d40ffd00 2023-07-14T19:28:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.997+0000 7f30d9828700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30c00a11d0 0x7f30c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:24.998+0000 7f30d9828700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30c00a11d0 0x7f30c00a3680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f30d40ffb60 tx=0x7f30c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.000+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f30d0097560 con 0x7f30d40ffd00 2023-07-14T19:28:25.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:24 smithi133 ceph-mon[128894]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:24 smithi133 ceph-mon[128894]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:24 smithi133 ceph-mon[128894]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:24 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3101396024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:25.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:24 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2357731656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:25.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.186+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30d402ca70 con 0x7f30d40ffd00 2023-07-14T19:28:25.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.187+0000 7f30caffd700 1 -- 172.21.15.82:0/611473500 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30d0004e60 con 0x7f30d40ffd00 2023-07-14T19:28:25.189 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:28:25.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30c00a11d0 msgr2=0x7f30c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30c00a11d0 0x7f30c00a3680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f30d40ffb60 tx=0x7f30c400b410 comp rx=0 tx=0).stop 2023-07-14T19:28:25.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 msgr2=0x7f30d4194e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4194e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f30d0004760 tx=0x7f30d0004790 comp rx=0 tx=0).stop 2023-07-14T19:28:25.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 shutdown_connections 2023-07-14T19:28:25.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d4100690 0x7f30d419be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30d40feb10 0x7f30d41948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30c00a11d0 0x7f30c00a3680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 --2- 172.21.15.82:0/611473500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30d40ffd00 0x7f30d4194e20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.192 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 >> 172.21.15.82:0/611473500 conn(0x7f30d40fa130 msgr2=0x7f30d4103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:25.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 shutdown_connections 2023-07-14T19:28:25.193 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.189+0000 7f30dba8c700 1 -- 172.21.15.82:0/611473500 wait complete. 2023-07-14T19:28:25.201 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:24 smithi082 ceph-mon[127461]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.201 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:24 smithi082 ceph-mon[127461]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.201 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:24 smithi082 ceph-mon[127461]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.202 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:24 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3101396024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:25.202 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:24 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2357731656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:25.203 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:28:25.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:24 smithi161 ceph-mon[129184]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:24 smithi161 ceph-mon[129184]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:24 smithi161 ceph-mon[129184]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-14T19:28:25.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:24 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3101396024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:25.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:24 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2357731656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:28:25.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.330+0000 7f8e10972700 1 -- 172.21.15.82:0/2310320286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c101800 msgr2=0x7f8e0c1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.332 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.330+0000 7f8e10972700 1 --2- 172.21.15.82:0/2310320286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c101800 0x7f8e0c1080c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400b0a0 tx=0x7f8e0400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 -- 172.21.15.82:0/2310320286 shutdown_connections 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 --2- 172.21.15.82:0/2310320286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c101800 0x7f8e0c1080c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 --2- 172.21.15.82:0/2310320286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c0ffc80 0x7f8e0c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 --2- 172.21.15.82:0/2310320286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e0c100e70 0x7f8e0c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 -- 172.21.15.82:0/2310320286 >> 172.21.15.82:0/2310320286 conn(0x7f8e0c0fb2a0 msgr2=0x7f8e0c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 -- 172.21.15.82:0/2310320286 shutdown_connections 2023-07-14T19:28:25.333 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.331+0000 7f8e10972700 1 -- 172.21.15.82:0/2310320286 wait complete. 2023-07-14T19:28:25.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 Processor -- start 2023-07-14T19:28:25.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 -- start start 2023-07-14T19:28:25.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e0c0ffc80 0x7f8e0c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 0x7f8e0c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c101800 0x7f8e0c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c19d5d0 con 0x7f8e0c100e70 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c19d710 con 0x7f8e0c101800 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e10972700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c19d8a0 con 0x7f8e0c0ffc80 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 0x7f8e0c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 0x7f8e0c195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42264/0 (socket says 172.21.15.82:42264) 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 -- 172.21.15.82:0/1827017617 learned_addr learned my addr 172.21.15.82:0/1827017617 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e0ad9d700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c101800 0x7f8e0c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e0a59c700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e0c0ffc80 0x7f8e0c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 -- 172.21.15.82:0/1827017617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e0c0ffc80 msgr2=0x7f8e0c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e0c0ffc80 0x7f8e0c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 -- 172.21.15.82:0/1827017617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c101800 msgr2=0x7f8e0c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c101800 0x7f8e0c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 -- 172.21.15.82:0/1827017617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04009d00 con 0x7f8e0c100e70 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e0ad9d700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c101800 0x7f8e0c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.332+0000 7f8e09d9b700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 0x7f8e0c195f90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000e9c0 tx=0x7f8e0000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.333+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e000097b0 con 0x7f8e0c100e70 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.333+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e00004cf0 con 0x7f8e0c100e70 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.333+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c19db80 con 0x7f8e0c100e70 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.333+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00005700 con 0x7f8e0c100e70 2023-07-14T19:28:25.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.333+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c19e040 con 0x7f8e0c100e70 2023-07-14T19:28:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.334+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0c1071a0 con 0x7f8e0c100e70 2023-07-14T19:28:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.335+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8e0000cbd0 con 0x7f8e0c100e70 2023-07-14T19:28:25.339 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.336+0000 7f8dfb7fe700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8df40a1150 0x7f8df40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.336+0000 7f8e0a59c700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8df40a1150 0x7f8df40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.336+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f8e00014070 con 0x7f8e0c100e70 2023-07-14T19:28:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.336+0000 7f8e0a59c700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8df40a1150 0x7f8df40a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c100cd0 tx=0x7f8dfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:25.340 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.339+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f8e0009a080 con 0x7f8e0c100e70 2023-07-14T19:28:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.529+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e0c02caf0 con 0x7f8e0c100e70 2023-07-14T19:28:25.531 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.530+0000 7f8dfb7fe700 1 -- 172.21.15.82:0/1827017617 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e00099c10 con 0x7f8e0c100e70 2023-07-14T19:28:25.532 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:28:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8df40a1150 msgr2=0x7f8df40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8df40a1150 0x7f8df40a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c100cd0 tx=0x7f8dfc009260 comp rx=0 tx=0).stop 2023-07-14T19:28:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 msgr2=0x7f8e0c195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 0x7f8e0c195f90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000e9c0 tx=0x7f8e0000ed80 comp rx=0 tx=0).stop 2023-07-14T19:28:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 shutdown_connections 2023-07-14T19:28:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e0c101800 0x7f8e0c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e0c0ffc80 0x7f8e0c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f8df40a1150 0x7f8df40a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 --2- 172.21.15.82:0/1827017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0c100e70 0x7f8e0c195f90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 >> 172.21.15.82:0/1827017617 conn(0x7f8e0c0fb2a0 msgr2=0x7f8e0c1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:25.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 shutdown_connections 2023-07-14T19:28:25.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.532+0000 7f8e10972700 1 -- 172.21.15.82:0/1827017617 wait complete. 2023-07-14T19:28:25.545 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:28:25.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 -- 172.21.15.82:0/4144878195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30600feb40 msgr2=0x7f30600fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 --2- 172.21.15.82:0/4144878195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30600feb40 0x7f30600fef50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3054009990 tx=0x7f3054009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 -- 172.21.15.82:0/4144878195 shutdown_connections 2023-07-14T19:28:25.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 --2- 172.21.15.82:0/4144878195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30601006c0 0x7f3060106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 --2- 172.21.15.82:0/4144878195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30600feb40 0x7f30600fef50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 --2- 172.21.15.82:0/4144878195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600ffd30 0x7f3060100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 -- 172.21.15.82:0/4144878195 >> 172.21.15.82:0/4144878195 conn(0x7f30600fa160 msgr2=0x7f30600fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:25.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 -- 172.21.15.82:0/4144878195 shutdown_connections 2023-07-14T19:28:25.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.666+0000 7f3066d31700 1 -- 172.21.15.82:0/4144878195 wait complete. 2023-07-14T19:28:25.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 Processor -- start 2023-07-14T19:28:25.672 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 -- start start 2023-07-14T19:28:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600feb40 0x7f30601947f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30600ffd30 0x7f3060194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 0x7f306019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306019c370 con 0x7f30601006c0 2023-07-14T19:28:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306019c4b0 con 0x7f30600feb40 2023-07-14T19:28:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3066d31700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306019c640 con 0x7f30600ffd30 2023-07-14T19:28:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 0x7f306019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 0x7f306019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42282/0 (socket says 172.21.15.82:42282) 2023-07-14T19:28:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 -- 172.21.15.82:0/1635924522 learned_addr learned my addr 172.21.15.82:0/1635924522 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:25.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f3064acd700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600feb40 0x7f30601947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f305ffff700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30600ffd30 0x7f3060194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 -- 172.21.15.82:0/1635924522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30600ffd30 msgr2=0x7f3060194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30600ffd30 0x7f3060194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 -- 172.21.15.82:0/1635924522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600feb40 msgr2=0x7f30601947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600feb40 0x7f30601947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 -- 172.21.15.82:0/1635924522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540095f0 con 0x7f30601006c0 2023-07-14T19:28:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.667+0000 7f30652ce700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 0x7f306019bda0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3058009ee0 tx=0x7f305800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.668+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305800ddd0 con 0x7f30601006c0 2023-07-14T19:28:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.668+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3058004020 con 0x7f30601006c0 2023-07-14T19:28:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.668+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30600687d0 con 0x7f30601006c0 2023-07-14T19:28:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.668+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580126c0 con 0x7f30601006c0 2023-07-14T19:28:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.668+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3060068ce0 con 0x7f30601006c0 2023-07-14T19:28:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.669+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306010a310 con 0x7f30601006c0 2023-07-14T19:28:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.670+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3058004490 con 0x7f30601006c0 2023-07-14T19:28:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.671+0000 7f305dffb700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30500a1220 0x7f30500a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.671+0000 7f3064acd700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30500a1220 0x7f30500a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.671+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f30580ce7f0 con 0x7f30601006c0 2023-07-14T19:28:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.671+0000 7f3064acd700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30500a1220 0x7f30500a36d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f305400b2e0 tx=0x7f3054005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.674+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3058098000 con 0x7f30601006c0 2023-07-14T19:28:25.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.909+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3060106090 con 0x7f30601006c0 2023-07-14T19:28:25.912 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.911+0000 7f305dffb700 1 -- 172.21.15.82:0/1635924522 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f3060106090 con 0x7f30601006c0 2023-07-14T19:28:25.912 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:25.912 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:28:19.848679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:28:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30500a1220 msgr2=0x7f30500a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30500a1220 0x7f30500a36d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f305400b2e0 tx=0x7f3054005a90 comp rx=0 tx=0).stop 2023-07-14T19:28:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 msgr2=0x7f306019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 0x7f306019bda0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3058009ee0 tx=0x7f305800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 shutdown_connections 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f30500a1220 0x7f30500a36d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30601006c0 0x7f306019bda0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600feb40 0x7f30601947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 --2- 172.21.15.82:0/1635924522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30600ffd30 0x7f3060194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 >> 172.21.15.82:0/1635924522 conn(0x7f30600fa160 msgr2=0x7f30601031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 shutdown_connections 2023-07-14T19:28:25.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:25.913+0000 7f3066d31700 1 -- 172.21.15.82:0/1635924522 wait complete. 2023-07-14T19:28:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 23 2023-07-14T19:28:25.926 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:28:26.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 -- 172.21.15.82:0/105529297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40feb20 msgr2=0x7f82b40fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 --2- 172.21.15.82:0/105529297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40feb20 0x7f82b40fef30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009990 tx=0x7f82a8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 -- 172.21.15.82:0/105529297 shutdown_connections 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 --2- 172.21.15.82:0/105529297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b41006a0 0x7f82b4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 --2- 172.21.15.82:0/105529297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b40feb20 0x7f82b40fef30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 --2- 172.21.15.82:0/105529297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b40ffd10 0x7f82b4100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 -- 172.21.15.82:0/105529297 >> 172.21.15.82:0/105529297 conn(0x7f82b40fa100 msgr2=0x7f82b40fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 -- 172.21.15.82:0/105529297 shutdown_connections 2023-07-14T19:28:26.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.046+0000 7f82bb515700 1 -- 172.21.15.82:0/105529297 wait complete. 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 Processor -- start 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 -- start start 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b40feb20 0x7f82b4194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b40ffd10 0x7f82b4194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 0x7f82b419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b419c2c0 con 0x7f82b41006a0 2023-07-14T19:28:26.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b419c400 con 0x7f82b40ffd10 2023-07-14T19:28:26.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82bb515700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b419c590 con 0x7f82b40feb20 2023-07-14T19:28:26.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 0x7f82b419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 0x7f82b419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42298/0 (socket says 172.21.15.82:42298) 2023-07-14T19:28:26.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 -- 172.21.15.82:0/1894565576 learned_addr learned my addr 172.21.15.82:0/1894565576 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b92b1700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b40feb20 0x7f82b4194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b8ab0700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b40ffd10 0x7f82b4194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 -- 172.21.15.82:0/1894565576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b40feb20 msgr2=0x7f82b4194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b40feb20 0x7f82b4194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 -- 172.21.15.82:0/1894565576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b40ffd10 msgr2=0x7f82b4194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b40ffd10 0x7f82b4194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.047+0000 7f82b9ab2700 1 -- 172.21.15.82:0/1894565576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80095f0 con 0x7f82b41006a0 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.048+0000 7f82b9ab2700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 0x7f82b419bcf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00ada0 tx=0x7f82ac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.048+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00f070 con 0x7f82b41006a0 2023-07-14T19:28:26.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.048+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82ac007b30 con 0x7f82b41006a0 2023-07-14T19:28:26.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.048+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b419c870 con 0x7f82b41006a0 2023-07-14T19:28:26.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.048+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac005280 con 0x7f82b41006a0 2023-07-14T19:28:26.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.048+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b419cd30 con 0x7f82b41006a0 2023-07-14T19:28:26.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.049+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b4106020 con 0x7f82b41006a0 2023-07-14T19:28:26.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.050+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f82ac00a090 con 0x7f82b41006a0 2023-07-14T19:28:26.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.050+0000 7f82a67fc700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f82a00a11a0 0x7f82a00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.051+0000 7f82b92b1700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f82a00a11a0 0x7f82a00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.051+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f82ac0cc320 con 0x7f82b41006a0 2023-07-14T19:28:26.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.051+0000 7f82b92b1700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f82a00a11a0 0x7f82a00a3650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f82a800b5c0 tx=0x7f82a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.054+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f82ac099650 con 0x7f82b41006a0 2023-07-14T19:28:26.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:25 smithi133 ceph-mon[128894]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:28:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:25 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/611473500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:26.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:25 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1827017617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:26.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:25 smithi082 ceph-mon[127461]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:28:26.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:25 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/611473500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:26.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:25 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1827017617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:26.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:25 smithi161 ceph-mon[129184]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2023-07-14T19:28:26.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:25 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/611473500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:28:26.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:25 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1827017617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:28:26.243 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.242+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82b402cb30 con 0x7f82b41006a0 2023-07-14T19:28:26.245 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.244+0000 7f82a67fc700 1 -- 172.21.15.82:0/1894565576 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22367 (secure 0 0 0) 0x7f82ac0991e0 con 0x7f82b41006a0 2023-07-14T19:28:26.246 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:26.247 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:27:05.776753+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:28:26.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f82a00a11a0 msgr2=0x7f82a00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f82a00a11a0 0x7f82a00a3650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f82a800b5c0 tx=0x7f82a8005960 comp rx=0 tx=0).stop 2023-07-14T19:28:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 msgr2=0x7f82b419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.249 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 0x7f82b419bcf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00ada0 tx=0x7f82ac00c330 comp rx=0 tx=0).stop 2023-07-14T19:28:26.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 shutdown_connections 2023-07-14T19:28:26.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f82a00a11a0 0x7f82a00a3650 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.250 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82b41006a0 0x7f82b419bcf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b40feb20 0x7f82b4194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 --2- 172.21.15.82:0/1894565576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b40ffd10 0x7f82b4194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 >> 172.21.15.82:0/1894565576 conn(0x7f82b40fa100 msgr2=0x7f82b4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 shutdown_connections 2023-07-14T19:28:26.251 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.246+0000 7f82bb515700 1 -- 172.21.15.82:0/1894565576 wait complete. 2023-07-14T19:28:26.260 DEBUG:tasks.fwd_scrub:Yielding 2023-07-14T19:28:26.260 INFO:teuthology.run_tasks:Running task exec... 2023-07-14T19:28:26.269 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T19:28:26.270 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:28:26.270 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-14T19:28:26.271 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-14T19:28:26.271 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T19:28:26.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T19:28:26.395 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 -- 172.21.15.82:0/1248470361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0068770 msgr2=0x7f52e01027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 --2- 172.21.15.82:0/1248470361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0068770 0x7f52e01027c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009990 tx=0x7f52d0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 -- 172.21.15.82:0/1248470361 shutdown_connections 2023-07-14T19:28:26.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 --2- 172.21.15.82:0/1248470361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52e0105810 0x7f52e0107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 --2- 172.21.15.82:0/1248470361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0068770 0x7f52e01027c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.396 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 --2- 172.21.15.82:0/1248470361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0102d00 0x7f52e0105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 -- 172.21.15.82:0/1248470361 >> 172.21.15.82:0/1248470361 conn(0x7f52e00fa130 msgr2=0x7f52e00fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 -- 172.21.15.82:0/1248470361 shutdown_connections 2023-07-14T19:28:26.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 -- 172.21.15.82:0/1248470361 wait complete. 2023-07-14T19:28:26.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 Processor -- start 2023-07-14T19:28:26.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.394+0000 7f52e627c700 1 -- start start 2023-07-14T19:28:26.398 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e627c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0068770 0x7f52e00fe760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e627c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 0x7f52e00feca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e627c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52e0105810 0x7f52e00ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e627c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e019f9c0 con 0x7f52e0102d00 2023-07-14T19:28:26.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e627c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e019fb00 con 0x7f52e0105810 2023-07-14T19:28:26.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e627c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e019fc40 con 0x7f52e0068770 2023-07-14T19:28:26.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 0x7f52e00feca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 0x7f52e00feca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42310/0 (socket says 172.21.15.82:42310) 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 -- 172.21.15.82:0/1757288499 learned_addr learned my addr 172.21.15.82:0/1757288499 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52e4819700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52e0105810 0x7f52e00ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52dffff700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0068770 0x7f52e00fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 -- 172.21.15.82:0/1757288499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0068770 msgr2=0x7f52e00fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0068770 0x7f52e00fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 -- 172.21.15.82:0/1757288499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52e0105810 msgr2=0x7f52e00ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52e0105810 0x7f52e00ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 -- 172.21.15.82:0/1757288499 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d00095f0 con 0x7f52e0102d00 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52dffff700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0068770 0x7f52e00fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.395+0000 7f52df7fe700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 0x7f52e00feca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f52d400d950 tx=0x7f52d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.396+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4009940 con 0x7f52e0102d00 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.396+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e019fed0 con 0x7f52e0102d00 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.396+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d4010460 con 0x7f52e0102d00 2023-07-14T19:28:26.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.396+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01a03a0 con 0x7f52e0102d00 2023-07-14T19:28:26.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.396+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400f620 con 0x7f52e0102d00 2023-07-14T19:28:26.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.397+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e00fbca0 con 0x7f52e0102d00 2023-07-14T19:28:26.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 -- 172.21.15.82:0/1371040262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 msgr2=0x7f209c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/1371040262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c107d30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f209400b0a0 tx=0x7f209400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 -- 172.21.15.82:0/1371040262 shutdown_connections 2023-07-14T19:28:26.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/1371040262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c107d30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/1371040262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/1371040262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f209c102fb0 0x7f209c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 -- 172.21.15.82:0/1371040262 >> 172.21.15.82:0/1371040262 conn(0x7f209c0fa130 msgr2=0x7f209c0fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 -- 172.21.15.82:0/1371040262 shutdown_connections 2023-07-14T19:28:26.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f20a0ca6700 1 -- 172.21.15.82:0/1371040262 wait complete. 2023-07-14T19:28:26.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.398+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f52d40108d0 con 0x7f52e0102d00 2023-07-14T19:28:26.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 Processor -- start 2023-07-14T19:28:26.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 -- start start 2023-07-14T19:28:26.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f52dd7fa700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f52c80a1150 0x7f52c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c0fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f52dffff700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f52c80a1150 0x7f52c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f52d40ccaf0 con 0x7f52e0102d00 2023-07-14T19:28:26.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f209c102fb0 0x7f209c0febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c0ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0ff920 con 0x7f209c1058d0 2023-07-14T19:28:26.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0ffa90 con 0x7f209c100690 2023-07-14T19:28:26.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f20a0ca6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c0ffc50 con 0x7f209c102fb0 2023-07-14T19:28:26.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c0ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c0fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c0fe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:38670/0 (socket says 172.21.15.82:38670) 2023-07-14T19:28:26.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c0ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42326/0 (socket says 172.21.15.82:42326) 2023-07-14T19:28:26.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 -- 172.21.15.82:0/2368160158 learned_addr learned my addr 172.21.15.82:0/2368160158 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f209c102fb0 0x7f209c0febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f52dffff700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f52c80a1150 0x7f52c80a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52d000b5c0 tx=0x7f52d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 -- 172.21.15.82:0/2368160158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f209c102fb0 msgr2=0x7f209c0febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f209c102fb0 0x7f209c0febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 -- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 msgr2=0x7f209c0ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c0ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209a59c700 1 -- 172.21.15.82:0/2368160158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2094009d00 con 0x7f209c100690 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.399+0000 7f209ad9d700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.400+0000 7f209a59c700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c0fe6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e9a0 tx=0x7f208c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.400+0000 7f208b7fe700 1 -- 172.21.15.82:0/2368160158 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c009790 con 0x7f209c100690 2023-07-14T19:28:26.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.400+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c0fff30 con 0x7f209c100690 2023-07-14T19:28:26.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.400+0000 7f208b7fe700 1 -- 172.21.15.82:0/2368160158 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208c010430 con 0x7f209c100690 2023-07-14T19:28:26.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.400+0000 7f208b7fe700 1 -- 172.21.15.82:0/2368160158 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c018680 con 0x7f209c100690 2023-07-14T19:28:26.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.400+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c100440 con 0x7f209c100690 2023-07-14T19:28:26.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.402+0000 7f208b7fe700 1 -- 172.21.15.82:0/2368160158 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f208c0105a0 con 0x7f209c100690 2023-07-14T19:28:26.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.403+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f52d4096300 con 0x7f52e0102d00 2023-07-14T19:28:26.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.403+0000 7f208b7fe700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f20840a1150 0x7f20840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.403+0000 7f208b7fe700 1 -- 172.21.15.82:0/2368160158 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f208c014070 con 0x7f209c100690 2023-07-14T19:28:26.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.403+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f20840a1150 0x7f20840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.403+0000 7f20ace3eb80 1 Processor -- start 2023-07-14T19:28:26.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 -- start start 2023-07-14T19:28:26.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f20840a1150 0x7f20840a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2090009c60 tx=0x7f2090009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 0x561f549d93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549da9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f548ddd30 con 0x561f549d8fe0 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f54930ac0 con 0x561f549da530 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f54929f50 con 0x561f549d9930 2023-07-14T19:28:26.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 0x561f549d93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 0x561f549d93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42330/0 (socket says 172.21.15.82:42330) 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 -- 172.21.15.82:0/658315319 learned_addr learned my addr 172.21.15.82:0/658315319 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f2099d9b700 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549d9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209ad9d700 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549da9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 -- 172.21.15.82:0/658315319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 msgr2=0x561f549d9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549d9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 -- 172.21.15.82:0/658315319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 msgr2=0x561f549da9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549da9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.404+0000 7f209a59c700 1 -- 172.21.15.82:0/658315319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f54757e60 con 0x561f549d8fe0 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f209a59c700 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 0x561f549d93f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f208c0963e0 tx=0x7f208c099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e421c36f3a2eca server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f2088ff9700 1 -- 172.21.15.82:0/658315319 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0a2330 con 0x561f549d8fe0 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f2088ff9700 1 -- 172.21.15.82:0/658315319 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208c098280 con 0x561f549d8fe0 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f2088ff9700 1 -- 172.21.15.82:0/658315319 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c0a2100 con 0x561f549d8fe0 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 -- 172.21.15.82:0/658315319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 msgr2=0x561f549d93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 0x561f549d93f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f208c0963e0 tx=0x7f208c099940 comp rx=0 tx=0).stop 2023-07-14T19:28:26.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 -- 172.21.15.82:0/658315319 shutdown_connections 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549da9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d8fe0 0x561f549d93f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/658315319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549d9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.405+0000 7f20ace3eb80 1 -- 172.21.15.82:0/658315319 >> 172.21.15.82:0/658315319 conn(0x561f549ce700 msgr2=0x561f549d3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 -- 172.21.15.82:0/658315319 shutdown_connections 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 -- 172.21.15.82:0/658315319 wait complete. 2023-07-14T19:28:26.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 -- start start 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 Processor -- start 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549e2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549da530 0x561f549e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f548c9b40 con 0x561f549d9930 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f548c9840 con 0x561f549d8fe0 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f547ad9a0 con 0x561f549da530 2023-07-14T19:28:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.406+0000 7f20ace3eb80 10 client.0 _ll_register_callbacks cb 0x561f548c77e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42338/0 (socket says 172.21.15.82:42338) 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 -- 172.21.15.82:0/3070222644 learned_addr learned my addr 172.21.15.82:0/3070222644 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f209a59c700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549e2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f209ad9d700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549da530 0x561f549e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 -- 172.21.15.82:0/3070222644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549da530 msgr2=0x561f549e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549da530 0x561f549e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 -- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 msgr2=0x561f549e2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549e2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 -- 172.21.15.82:0/3070222644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c100440 con 0x561f549d9930 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2099d9b700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e2fb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2090003900 tx=0x7f20900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d12eb5475e6a02d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2081ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2081ffb700 1 -- 172.21.15.82:0/3070222644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090022070 con 0x561f549d9930 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f54883570 con 0x561f549d9930 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2081ffb700 1 -- 172.21.15.82:0/3070222644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f209001d620 con 0x561f549d9930 2023-07-14T19:28:26.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f2081ffb700 1 -- 172.21.15.82:0/3070222644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f209001d7e0 con 0x561f549d9930 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f20ace3eb80 10 client.15384 fetch_fsmap learned FSMap version 23 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.407+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561f54740ce0 con 0x561f549d9930 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f2081ffb700 1 -- 172.21.15.82:0/3070222644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f54883570 con 0x561f549d9930 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f2081ffb700 1 -- 172.21.15.82:0/3070222644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2090032b90 con 0x561f549d9930 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f20ace3eb80 10 client.15384 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f20ace3eb80 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:26.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f20ace3eb80 20 client.15384 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f549ee900 0x561f549f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f20ace3eb80 4 client.15384 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f549a7e20 con 0x561f549ee900 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.408+0000 7f209a59c700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f549ee900 0x561f549f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f2081ffb700 10 client.15384 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f209a59c700 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f549ee900 0x561f549f0d10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428d493889505eb2 server_cookie=baf0822c1c25334f in_seq=0 out_seq=0 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f2081ffb700 1 -- 172.21.15.82:0/3070222644 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561f549a7e20 con 0x561f549ee900 2023-07-14T19:28:26.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f2081ffb700 10 client.15384 handle_command_reply: tid=0 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 client.15384 shutdown 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 msgr2=0x561f549e2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e2fb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2090003900 tx=0x7f20900047c0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 shutdown_connections 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f549ee900 0x561f549f0d10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549da530 0x561f549e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549e2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f2081ffb700 0 client.15384 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/3070222644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e2fb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.409+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 >> 172.21.15.82:0/3070222644 conn(0x561f549ce700 msgr2=0x561f549d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.410+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 shutdown_connections 2023-07-14T19:28:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.410+0000 7f20ace3eb80 1 -- 172.21.15.82:0/3070222644 wait complete. 2023-07-14T19:28:26.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.410+0000 7f20ace3eb80 20 client.15384 trim_cache size 0 max 16384 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 Processor -- start 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 -- start start 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 0x561f549dd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f54883570 con 0x561f549da530 2023-07-14T19:28:26.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f549ce320 con 0x561f549d8fe0 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.426+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f549e1d70 con 0x561f549d9930 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 0x561f549dd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 0x561f549dd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42348/0 (socket says 172.21.15.82:42348) 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 -- 172.21.15.82:0/2282437818 learned_addr learned my addr 172.21.15.82:0/2282437818 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209a59c700 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.430 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 -- 172.21.15.82:0/2282437818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 msgr2=0x561f549e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 -- 172.21.15.82:0/2282437818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 msgr2=0x561f549ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 -- 172.21.15.82:0/2282437818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f5477e2d0 con 0x561f549da530 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f209ad9d700 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 0x561f549dd410 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2094015000 tx=0x7f2094009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca16e99c245e742 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f20897fa700 1 -- 172.21.15.82:0/2282437818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400e040 con 0x561f549da530 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f20897fa700 1 -- 172.21.15.82:0/2282437818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2094003a00 con 0x561f549da530 2023-07-14T19:28:26.431 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.427+0000 7f20897fa700 1 -- 172.21.15.82:0/2282437818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094027800 con 0x561f549da530 2023-07-14T19:28:26.434 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2282437818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 msgr2=0x561f549dd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 0x561f549dd410 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2094015000 tx=0x7f2094009460 comp rx=0 tx=0).stop 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2282437818 shutdown_connections 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549da530 0x561f549dd410 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549d8fe0 0x561f549ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2282437818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d9930 0x561f549e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2282437818 >> 172.21.15.82:0/2282437818 conn(0x561f549ce700 msgr2=0x561f549d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.435 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2282437818 shutdown_connections 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2282437818 wait complete. 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 -- start start 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.428+0000 7f20ace3eb80 1 Processor -- start 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d8fe0 0x561f549e7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.436 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f549e8ca0 con 0x561f549d9930 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f549e8e10 con 0x561f549da530 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f549e8fd0 con 0x561f549d8fe0 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 10 client.0 _ll_register_callbacks cb 0x7f20900044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42352/0 (socket says 172.21.15.82:42352) 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 -- 172.21.15.82:0/2896543060 learned_addr learned my addr 172.21.15.82:0/2896543060 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f209a59c700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d8fe0 0x561f549e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.437 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f209ad9d700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 -- 172.21.15.82:0/2896543060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d8fe0 msgr2=0x561f549e7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d8fe0 0x561f549e7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 -- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 msgr2=0x561f549e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549e8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 -- 172.21.15.82:0/2896543060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c108920 con 0x561f549d9930 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f209ad9d700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2099d9b700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e7750 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f2090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96247e3a9326d1f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.438 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2081ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2081ffb700 1 -- 172.21.15.82:0/2896543060 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090022040 con 0x561f549d9930 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2081ffb700 1 -- 172.21.15.82:0/2896543060 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f209001d830 con 0x561f549d9930 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f208c098280 con 0x561f549d9930 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f2081ffb700 1 -- 172.21.15.82:0/2896543060 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2090027610 con 0x561f549d9930 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 10 client.15390 fetch_fsmap learned FSMap version 23 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.429+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561f54756330 con 0x561f549d9930 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f2081ffb700 1 -- 172.21.15.82:0/2896543060 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208c098280 con 0x561f549d9930 2023-07-14T19:28:26.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f2081ffb700 1 -- 172.21.15.82:0/2896543060 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20900329d0 con 0x561f549d9930 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f20ace3eb80 10 client.15390 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f20ace3eb80 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f20ace3eb80 20 client.15390 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f54a44a80 0x561f54a46e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f20ace3eb80 4 client.15390 mds_command: new command op to 24524 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561f54922f30 con 0x561f54a44a80 2023-07-14T19:28:26.440 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f209a59c700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f54a44a80 0x561f54a46e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f2081ffb700 10 client.15390 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.430+0000 7f209a59c700 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f54a44a80 0x561f54a46e90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5211575d98b9744e server_cookie=db4313053422c17d in_seq=0 out_seq=0 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f2081ffb700 1 -- 172.21.15.82:0/2896543060 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561f54922f30 con 0x561f54a44a80 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f2081ffb700 10 client.15390 handle_command_reply: tid=0 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 client.15390 shutdown 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 msgr2=0x561f549e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e7750 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f2090005880 comp rx=0 tx=0).stop 2023-07-14T19:28:26.441 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 shutdown_connections 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f549da530 0x561f549e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561f54a44a80 0x561f54a46e90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f549d8fe0 0x561f549e7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 --2- 172.21.15.82:0/2896543060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f549d9930 0x561f549e7750 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f2081ffb700 0 client.15390 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 >> 172.21.15.82:0/2896543060 conn(0x561f549ce700 msgr2=0x561f549ea7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.442 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 shutdown_connections 2023-07-14T19:28:26.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 1 -- 172.21.15.82:0/2896543060 wait complete. 2023-07-14T19:28:26.443 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20ace3eb80 20 client.15390 trim_cache size 0 max 16384 2023-07-14T19:28:26.443 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:26.443 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2023-07-14T19:28:26.443 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "140cd9da-3f58-4edb-afb7-e7081cc5bcf3", 2023-07-14T19:28:26.444 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2023-07-14T19:28:26.444 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:26.444 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f20840a1150 msgr2=0x7f20840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f20840a1150 0x7f20840a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2090009c60 tx=0x7f2090009260 comp rx=0 tx=0).stop 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 msgr2=0x7f209c0fe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c0fe6a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e9a0 tx=0x7f208c00ecb0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 shutdown_connections 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.432+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f20840a1150 0x7f20840a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.433+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f209c1058d0 0x7f209c0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.433+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c100690 0x7f209c0fe6a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.445 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.433+0000 7f20a0ca6700 1 --2- 172.21.15.82:0/2368160158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f209c102fb0 0x7f209c0febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.433+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 >> 172.21.15.82:0/2368160158 conn(0x7f209c0fa130 msgr2=0x7f209c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.433+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 shutdown_connections 2023-07-14T19:28:26.446 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.433+0000 7f20a0ca6700 1 -- 172.21.15.82:0/2368160158 wait complete. 2023-07-14T19:28:26.447 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T19:28:26.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 -- 172.21.15.82:0/3788564878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 msgr2=0x7fd220101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 --2- 172.21.15.82:0/3788564878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220101260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd214009990 tx=0x7fd214009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 -- 172.21.15.82:0/3788564878 shutdown_connections 2023-07-14T19:28:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 --2- 172.21.15.82:0/3788564878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2201017a0 0x7fd220108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 --2- 172.21.15.82:0/3788564878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2200ffc20 0x7fd220100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 --2- 172.21.15.82:0/3788564878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220101260 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 -- 172.21.15.82:0/3788564878 >> 172.21.15.82:0/3788564878 conn(0x7fd2200fb260 msgr2=0x7fd2200fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 -- 172.21.15.82:0/3788564878 shutdown_connections 2023-07-14T19:28:26.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.570+0000 7fd2252be700 1 -- 172.21.15.82:0/3788564878 wait complete. 2023-07-14T19:28:26.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.571+0000 7fd2252be700 1 Processor -- start 2023-07-14T19:28:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.571+0000 7fd2252be700 1 -- start start 2023-07-14T19:28:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.571+0000 7fd2252be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2200ffc20 0x7fd220195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.571+0000 7fd2252be700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd2252be700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2201017a0 0x7fd22019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd2252be700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22019d580 con 0x7fd220100e10 2023-07-14T19:28:26.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd2252be700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22019d6c0 con 0x7fd2200ffc20 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd2252be700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22019d850 con 0x7fd2201017a0 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42366/0 (socket says 172.21.15.82:42366) 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 -- 172.21.15.82:0/4265585878 learned_addr learned my addr 172.21.15.82:0/4265585878 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2201017a0 0x7fd22019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21effd700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2200ffc20 0x7fd220195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 -- 172.21.15.82:0/4265585878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2201017a0 msgr2=0x7fd22019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2201017a0 0x7fd22019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 -- 172.21.15.82:0/4265585878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2200ffc20 msgr2=0x7fd220195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2200ffc20 0x7fd220195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 -- 172.21.15.82:0/4265585878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2140095f0 con 0x7fd220100e10 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21effd700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2200ffc20 0x7fd220195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220195f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd214004760 tx=0x7fd214004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd207fff700 1 -- 172.21.15.82:0/4265585878 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21401d070 con 0x7fd220100e10 2023-07-14T19:28:26.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22019dad0 con 0x7fd220100e10 2023-07-14T19:28:26.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd207fff700 1 -- 172.21.15.82:0/4265585878 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2140041d0 con 0x7fd220100e10 2023-07-14T19:28:26.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd207fff700 1 -- 172.21.15.82:0/4265585878 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2140053a0 con 0x7fd220100e10 2023-07-14T19:28:26.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.572+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22019dff0 con 0x7fd220100e10 2023-07-14T19:28:26.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.575+0000 7fd207fff700 1 -- 172.21.15.82:0/4265585878 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd214005500 con 0x7fd220100e10 2023-07-14T19:28:26.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.575+0000 7fd207fff700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2080a1150 0x7fd2080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.575+0000 7fd207fff700 1 -- 172.21.15.82:0/4265585878 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fd2140cdae0 con 0x7fd220100e10 2023-07-14T19:28:26.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.575+0000 7fd21effd700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2080a1150 0x7fd2080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.584 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd21effd700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2080a1150 0x7fd2080a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd220100c70 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 Processor -- start 2023-07-14T19:28:26.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 -- start start 2023-07-14T19:28:26.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 0x55a62e903370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e903ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.585 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e9044b0 0x55a62e904920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e807cb0 con 0x55a62e902f60 2023-07-14T19:28:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e85aa40 con 0x55a62e9038b0 2023-07-14T19:28:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd231456b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e853ed0 con 0x55a62e9044b0 2023-07-14T19:28:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd21effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 0x55a62e903370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd21effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 0x55a62e903370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42372/0 (socket says 172.21.15.82:42372) 2023-07-14T19:28:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd21effd700 1 -- 172.21.15.82:0/1567911138 learned_addr learned my addr 172.21.15.82:0/1567911138 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.576+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e9044b0 0x55a62e904920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e903ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21effd700 1 -- 172.21.15.82:0/1567911138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e9044b0 msgr2=0x55a62e904920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21effd700 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e9044b0 0x55a62e904920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21effd700 1 -- 172.21.15.82:0/1567911138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 msgr2=0x55a62e903ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21effd700 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e903ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21effd700 1 -- 172.21.15.82:0/1567911138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62e681de0 con 0x55a62e902f60 2023-07-14T19:28:26.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd21effd700 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 0x55a62e903370 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd21000d7a0 tx=0x7fd210010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d017cd0654d87a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd2057fa700 1 -- 172.21.15.82:0/1567911138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210014640 con 0x55a62e902f60 2023-07-14T19:28:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd2057fa700 1 -- 172.21.15.82:0/1567911138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd210014c60 con 0x55a62e902f60 2023-07-14T19:28:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.577+0000 7fd2057fa700 1 -- 172.21.15.82:0/1567911138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21001fba0 con 0x55a62e902f60 2023-07-14T19:28:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- 172.21.15.82:0/1567911138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 msgr2=0x55a62e903370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 0x55a62e903370 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd21000d7a0 tx=0x7fd210010000 comp rx=0 tx=0).stop 2023-07-14T19:28:26.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- 172.21.15.82:0/1567911138 shutdown_connections 2023-07-14T19:28:26.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e9044b0 0x55a62e904920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e902f60 0x55a62e903370 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- 172.21.15.82:0/1567911138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e903ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- 172.21.15.82:0/1567911138 >> 172.21.15.82:0/1567911138 conn(0x55a62e8f8680 msgr2=0x55a62e8fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- 172.21.15.82:0/1567911138 shutdown_connections 2023-07-14T19:28:26.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- 172.21.15.82:0/1567911138 wait complete. 2023-07-14T19:28:26.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- start start 2023-07-14T19:28:26.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 Processor -- start 2023-07-14T19:28:26.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e90c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e90cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e913f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e7f3ac0 con 0x55a62e9044b0 2023-07-14T19:28:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e7f37c0 con 0x55a62e9038b0 2023-07-14T19:28:26.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e6d7920 con 0x55a62e902f60 2023-07-14T19:28:26.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd231456b80 10 client.0 _ll_register_callbacks cb 0x55a62e7f1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:26.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.578+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e913f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e913f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42382/0 (socket says 172.21.15.82:42382) 2023-07-14T19:28:26.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 -- 172.21.15.82:0/894031555 learned_addr learned my addr 172.21.15.82:0/894031555 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21effd700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e90c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e90cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 -- 172.21.15.82:0/894031555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 msgr2=0x55a62e90c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e90c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 -- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 msgr2=0x55a62e90cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e90cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 -- 172.21.15.82:0/894031555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21000e040 con 0x55a62e9044b0 2023-07-14T19:28:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e913f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd21800ae90 tx=0x7fd218012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfe19cd68ab1113 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd202ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd202ff5700 1 -- 172.21.15.82:0/894031555 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800f040 con 0x55a62e9044b0 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd202ff5700 1 -- 172.21.15.82:0/894031555 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fd218012ca0 con 0x55a62e9044b0 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a62e7ad4f0 con 0x55a62e9044b0 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd202ff5700 1 -- 172.21.15.82:0/894031555 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd218012e60 con 0x55a62e9044b0 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd231456b80 10 client.15402 fetch_fsmap learned FSMap version 23 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a62e66ac60 con 0x55a62e9044b0 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.579+0000 7fd202ff5700 1 -- 172.21.15.82:0/894031555 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a62e7ad4f0 con 0x55a62e9044b0 2023-07-14T19:28:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd202ff5700 1 -- 172.21.15.82:0/894031555 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2180083f0 con 0x55a62e9044b0 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd231456b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd231456b80 10 client.15402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd231456b80 20 client.15402 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd231456b80 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e918770 0x55a62e91ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd231456b80 4 client.15402 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a62e8d1da0 con 0x55a62e918770 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd21effd700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e918770 0x55a62e91ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd202ff5700 10 client.15402 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.580+0000 7fd21effd700 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e918770 0x55a62e91ab80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6aede9fea7b968a server_cookie=1ed4a229b872bf6b in_seq=0 out_seq=0 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd202ff5700 1 -- 172.21.15.82:0/894031555 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a62e8d1da0 con 0x55a62e918770 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd202ff5700 10 client.15402 handle_command_reply: tid=0 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 client.15402 shutdown 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 msgr2=0x55a62e913f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e913f50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd21800ae90 tx=0x7fd218012510 comp rx=0 tx=0).stop 2023-07-14T19:28:26.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 shutdown_connections 2023-07-14T19:28:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e913f50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e918770 0x55a62e91ab80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e90c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 --2- 172.21.15.82:0/894031555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e90cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd202ff5700 0 client.15402 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 >> 172.21.15.82:0/894031555 conn(0x55a62e8f8680 msgr2=0x55a62e8fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 shutdown_connections 2023-07-14T19:28:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 1 -- 172.21.15.82:0/894031555 wait complete. 2023-07-14T19:28:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.581+0000 7fd231456b80 20 client.15402 trim_cache size 0 max 16384 2023-07-14T19:28:26.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 Processor -- start 2023-07-14T19:28:26.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 -- start start 2023-07-14T19:28:26.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e904880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e913e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e90a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e90bed0 con 0x55a62e9044b0 2023-07-14T19:28:26.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e8fe8e0 con 0x55a62e9038b0 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd231456b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e90a6a0 con 0x55a62e902f60 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.598+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e90a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e90a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42390/0 (socket says 172.21.15.82:42390) 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 -- 172.21.15.82:0/1314793315 learned_addr learned my addr 172.21.15.82:0/1314793315 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21effd700 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e904880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e913e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 -- 172.21.15.82:0/1314793315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 msgr2=0x55a62e904880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e904880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 -- 172.21.15.82:0/1314793315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 msgr2=0x55a62e913e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e913e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 -- 172.21.15.82:0/1314793315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62e6a8250 con 0x55a62e9044b0 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21effd700 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e904880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e90a8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd21800a400 tx=0x7fd218012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9958ae8b34f63faa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.609 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd205ffb700 1 -- 172.21.15.82:0/1314793315 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800f040 con 0x55a62e9044b0 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd205ffb700 1 -- 172.21.15.82:0/1314793315 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd218010040 con 0x55a62e9044b0 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.599+0000 7fd205ffb700 1 -- 172.21.15.82:0/1314793315 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21801f440 con 0x55a62e9044b0 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 -- 172.21.15.82:0/1314793315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 msgr2=0x55a62e90a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e90a8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd21800a400 tx=0x7fd218012910 comp rx=0 tx=0).stop 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 -- 172.21.15.82:0/1314793315 shutdown_connections 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e90a8f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.610 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e904880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 --2- 172.21.15.82:0/1314793315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e913e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 -- 172.21.15.82:0/1314793315 >> 172.21.15.82:0/1314793315 conn(0x55a62e8f8680 msgr2=0x55a62e90b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 -- 172.21.15.82:0/1314793315 shutdown_connections 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 -- 172.21.15.82:0/1314793315 wait complete. 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 -- start start 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.600+0000 7fd231456b80 1 Processor -- start 2023-07-14T19:28:26.611 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e913e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e912460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e912a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e913160 con 0x55a62e9044b0 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e9132d0 con 0x55a62e9038b0 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e96c430 con 0x55a62e902f60 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 10 client.0 _ll_register_callbacks cb 0x7fd21800a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:26.612 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e912a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e912a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42396/0 (socket says 172.21.15.82:42396) 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 -- 172.21.15.82:0/3532278805 learned_addr learned my addr 172.21.15.82:0/3532278805 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21effd700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e913e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21e7fc700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e912460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 -- 172.21.15.82:0/3532278805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 msgr2=0x55a62e913e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e913e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 -- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 msgr2=0x55a62e912460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:26.613 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e912460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 -- 172.21.15.82:0/3532278805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a62e66ac60 con 0x55a62e9044b0 2023-07-14T19:28:26.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd21f7fe700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e912a90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd21800ae90 tx=0x7fd218012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2532c6abbeee3c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:26.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd202ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:26.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd202ff5700 1 -- 172.21.15.82:0/3532278805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800f040 con 0x55a62e9044b0 2023-07-14T19:28:26.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd202ff5700 1 -- 172.21.15.82:0/3532278805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fd218004c90 con 0x55a62e9044b0 2023-07-14T19:28:26.614 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd210014c60 con 0x55a62e9044b0 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd202ff5700 1 -- 172.21.15.82:0/3532278805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd218004e90 con 0x55a62e9044b0 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.601+0000 7fd231456b80 10 client.15408 fetch_fsmap learned FSMap version 23 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a62e6802b0 con 0x55a62e9044b0 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd202ff5700 1 -- 172.21.15.82:0/3532278805 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd218045020 con 0x55a62e9044b0 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd202ff5700 1 -- 172.21.15.82:0/3532278805 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2180417a0 con 0x55a62e9044b0 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 10 client.15408 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T19:28:26.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 20 client.15408 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e96d9d0 0x55a62e96fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 4 client.15408 mds_command: new command op to 24524 tid=0[{"prefix": "scrub status"}] 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a62e84ceb0 con 0x55a62e96d9d0 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd21effd700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e96d9d0 0x55a62e96fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd202ff5700 10 client.15408 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.602+0000 7fd21effd700 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e96d9d0 0x55a62e96fde0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e4d721ce3fafa9 server_cookie=f2df3b40efdd2f0d in_seq=0 out_seq=0 2023-07-14T19:28:26.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd202ff5700 1 -- 172.21.15.82:0/3532278805 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a62e84ceb0 con 0x55a62e96d9d0 2023-07-14T19:28:26.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd202ff5700 10 client.15408 handle_command_reply: tid=0 2023-07-14T19:28:26.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 client.15408 shutdown 2023-07-14T19:28:26.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 msgr2=0x55a62e912a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e912a90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd21800ae90 tx=0x7fd218012510 comp rx=0 tx=0).stop 2023-07-14T19:28:26.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 shutdown_connections 2023-07-14T19:28:26.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a62e9044b0 0x55a62e912a90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a62e96d9d0 0x55a62e96fde0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a62e902f60 0x55a62e913e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd202ff5700 0 client.15408 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:26.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 --2- 172.21.15.82:0/3532278805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a62e9038b0 0x55a62e912460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 >> 172.21.15.82:0/3532278805 conn(0x55a62e8f8680 msgr2=0x55a62e90a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 shutdown_connections 2023-07-14T19:28:26.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 1 -- 172.21.15.82:0/3532278805 wait complete. 2023-07-14T19:28:26.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd231456b80 20 client.15408 trim_cache size 0 max 16384 2023-07-14T19:28:26.619 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:26.620 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T19:28:26.620 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-07-14T19:28:26.620 INFO:teuthology.orchestra.run.smithi082.stdout: "140cd9da-3f58-4edb-afb7-e7081cc5bcf3": { 2023-07-14T19:28:26.620 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-07-14T19:28:26.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "140cd9da-3f58-4edb-afb7-e7081cc5bcf3", 2023-07-14T19:28:26.620 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-07-14T19:28:26.621 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:28:26.621 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:28:26.621 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:26.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2080a1150 msgr2=0x7fd2080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd2252be700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2080a1150 0x7fd2080a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd220100c70 tx=0x7fd210008040 comp rx=0 tx=0).stop 2023-07-14T19:28:26.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 msgr2=0x7fd220195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.603+0000 7fd2252be700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220195f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd214004760 tx=0x7fd214004790 comp rx=0 tx=0).stop 2023-07-14T19:28:26.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 shutdown_connections 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd2201017a0 0x7fd22019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2200ffc20 0x7fd220195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd2080a1150 0x7fd2080a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 --2- 172.21.15.82:0/4265585878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd220100e10 0x7fd220195f40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 >> 172.21.15.82:0/4265585878 conn(0x7fd2200fb260 msgr2=0x7fd220104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 shutdown_connections 2023-07-14T19:28:26.623 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.604+0000 7fd2252be700 1 -- 172.21.15.82:0/4265585878 wait complete. 2023-07-14T19:28:26.624 INFO:tasks.cephfs.filesystem:scrub status for tag:140cd9da-3f58-4edb-afb7-e7081cc5bcf3 - {'path': '/', 'tag': '140cd9da-3f58-4edb-afb7-e7081cc5bcf3', 'options': 'recursive,force'} 2023-07-14T19:28:26.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.667+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f52e002c8e0 con 0x7f52e0102d00 2023-07-14T19:28:26.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.887+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f52d40105d0 con 0x7f52e0102d00 2023-07-14T19:28:26.889 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.888+0000 7f52dd7fa700 1 -- 172.21.15.82:0/1757288499 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f52d408e980 con 0x7f52e0102d00 2023-07-14T19:28:26.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f52c80a1150 msgr2=0x7f52c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f52c80a1150 0x7f52c80a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f52d000b5c0 tx=0x7f52d0005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 msgr2=0x7f52e00feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:26.892 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 0x7f52e00feca0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f52d400d950 tx=0x7f52d400dc60 comp rx=0 tx=0).stop 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 shutdown_connections 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52e0105810 0x7f52e00ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52e0068770 0x7f52e00fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f52c80a1150 0x7f52c80a3600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 --2- 172.21.15.82:0/1757288499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0102d00 0x7f52e00feca0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.890+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 >> 172.21.15.82:0/1757288499 conn(0x7f52e00fa130 msgr2=0x7f52e0101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:26.893 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.891+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 shutdown_connections 2023-07-14T19:28:26.894 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:26.891+0000 7f52e627c700 1 -- 172.21.15.82:0/1757288499 wait complete. 2023-07-14T19:28:26.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-14T19:28:27.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 -- 172.21.15.82:0/2913905778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e0100720 msgr2=0x7fd3e0106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 --2- 172.21.15.82:0/2913905778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e0100720 0x7fd3e0106fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800b0a0 tx=0x7fd3d800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 -- 172.21.15.82:0/2913905778 shutdown_connections 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 --2- 172.21.15.82:0/2913905778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e0100720 0x7fd3e0106fc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 --2- 172.21.15.82:0/2913905778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e00ff060 0x7fd3e00ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 --2- 172.21.15.82:0/2913905778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e00ffd70 0x7fd3e01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 -- 172.21.15.82:0/2913905778 >> 172.21.15.82:0/2913905778 conn(0x7fd3e00f98e0 msgr2=0x7fd3e00fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 -- 172.21.15.82:0/2913905778 shutdown_connections 2023-07-14T19:28:27.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 -- 172.21.15.82:0/2913905778 wait complete. 2023-07-14T19:28:27.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.032+0000 7fd3e7780700 1 Processor -- start 2023-07-14T19:28:27.037 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 -- start start 2023-07-14T19:28:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e00ff060 0x7fd3e0194a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 0x7fd3e0194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e0100720 0x7fd3e0195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0196470 con 0x7fd3e00ffd70 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e01965e0 con 0x7fd3e00ff060 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e7780700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e01967a0 con 0x7fd3e0100720 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 0x7fd3e0194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 0x7fd3e0194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42432/0 (socket says 172.21.15.82:42432) 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 -- 172.21.15.82:0/3694634635 learned_addr learned my addr 172.21.15.82:0/3694634635 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e551c700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e00ff060 0x7fd3e0194a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e5d1d700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e0100720 0x7fd3e0195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 -- 172.21.15.82:0/3694634635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e0100720 msgr2=0x7fd3e0195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e0100720 0x7fd3e0195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 -- 172.21.15.82:0/3694634635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e00ff060 msgr2=0x7fd3e0194a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e00ff060 0x7fd3e0194a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 -- 172.21.15.82:0/3694634635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d8009d00 con 0x7fd3e00ffd70 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.033+0000 7fd3e4d1b700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 0x7fd3e0194f90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d980 tx=0x7fd3d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.034+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0004020 con 0x7fd3e00ffd70 2023-07-14T19:28:27.040 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.034+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d000f690 con 0x7fd3e00ffd70 2023-07-14T19:28:27.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.034+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e0196a80 con 0x7fd3e00ffd70 2023-07-14T19:28:27.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.034+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0003a60 con 0x7fd3e00ffd70 2023-07-14T19:28:27.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.034+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e019d240 con 0x7fd3e00ffd70 2023-07-14T19:28:27.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.035+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e004e420 con 0x7fd3e00ffd70 2023-07-14T19:28:27.041 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.036+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7fd3d000fb00 con 0x7fd3e00ffd70 2023-07-14T19:28:27.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.036+0000 7fd3d67fc700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 0x7fd3cc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.036+0000 7fd3e551c700 1 -- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 msgr2=0x7fd3cc0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:27.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.036+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fd3d00cc750 con 0x7fd3e00ffd70 2023-07-14T19:28:27.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.036+0000 7fd3e551c700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 0x7fd3cc0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:28:27.042 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.040+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fd3d0095f60 con 0x7fd3e00ffd70 2023-07-14T19:28:27.137 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:26 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1635924522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:27.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:26 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1894565576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:27.138 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:26 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1757288499' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-14T19:28:27.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:26 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: ignoring --setuser ceph since I am not root 2023-07-14T19:28:27.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:26 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: ignoring --setgroup ceph since I am not root 2023-07-14T19:28:27.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:26 smithi161 ceph-mgr[131168]: -- 172.21.15.161:0/2726582667 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621d927f1e0 con 0x5621d9254800 2023-07-14T19:28:27.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.056+0000 7f89d5d64200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:28:27.140 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.135+0000 7f89d5d64200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:28:27.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1635924522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:27.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1894565576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:27.141 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:26 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1757288499' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-14T19:28:27.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:26 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1635924522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:27.142 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:26 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1894565576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:28:27.143 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:26 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1757288499' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-14T19:28:27.143 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:26 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: ignoring --setuser ceph since I am not root 2023-07-14T19:28:27.143 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:26 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: ignoring --setgroup ceph since I am not root 2023-07-14T19:28:27.144 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.055+0000 7fad3b7ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:28:27.144 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.137+0000 7fad3b7ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:28:27.144 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:26 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: ignoring --setuser ceph since I am not root 2023-07-14T19:28:27.145 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:26 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: ignoring --setgroup ceph since I am not root 2023-07-14T19:28:27.145 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:26 smithi133 ceph-mgr[130367]: -- 172.21.15.133:0/1987722577 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55adbe70d1e0 con 0x55adbe6e3000 2023-07-14T19:28:27.145 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.056+0000 7fd3007a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-14T19:28:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.237+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd3e0105a30 con 0x7fd3e00ffd70 2023-07-14T19:28:27.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.237+0000 7fd3e551c700 1 -- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 msgr2=0x7fd3cc0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:27.239 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.237+0000 7fd3e551c700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 0x7fd3cc0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:28:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.243+0000 7fd3d67fc700 1 -- 172.21.15.82:0/3694634635 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd3d0099580 con 0x7fd3e00ffd70 2023-07-14T19:28:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 msgr2=0x7fd3cc0a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:28:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 0x7fd3cc0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 msgr2=0x7fd3e0194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 0x7fd3e0194f90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d980 tx=0x7fd3d000dc90 comp rx=0 tx=0).stop 2023-07-14T19:28:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 shutdown_connections 2023-07-14T19:28:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e0100720 0x7fd3e0195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e00ff060 0x7fd3e0194a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fd3cc0a11a0 0x7fd3cc0a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 --2- 172.21.15.82:0/3694634635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3e00ffd70 0x7fd3e0194f90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 >> 172.21.15.82:0/3694634635 conn(0x7fd3e00f98e0 msgr2=0x7fd3e00fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 shutdown_connections 2023-07-14T19:28:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.245+0000 7fd3e7780700 1 -- 172.21.15.82:0/3694634635 wait complete. 2023-07-14T19:28:27.259 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-14T19:28:27.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 -- 172.21.15.82:0/2397799914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 msgr2=0x7fb984072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 --2- 172.21.15.82:0/2397799914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb984072a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb974009990 tx=0x7fb974009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 -- 172.21.15.82:0/2397799914 shutdown_connections 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 --2- 172.21.15.82:0/2397799914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb984107af0 0x7fb984109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 --2- 172.21.15.82:0/2397799914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb984072a50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 --2- 172.21.15.82:0/2397799914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984073020 0x7fb984073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 -- 172.21.15.82:0/2397799914 >> 172.21.15.82:0/2397799914 conn(0x7fb9840fb260 msgr2=0x7fb9840fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 -- 172.21.15.82:0/2397799914 shutdown_connections 2023-07-14T19:28:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.385+0000 7fb988ed6700 1 -- 172.21.15.82:0/2397799914 wait complete. 2023-07-14T19:28:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 Processor -- start 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 -- start start 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb984073020 0x7fb984197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb9841982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984107af0 0x7fb9841990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841997a0 con 0x7fb984074600 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984199910 con 0x7fb984073020 2023-07-14T19:28:27.390 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb988ed6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb984199ad0 con 0x7fb984107af0 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb9841982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb9841982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42478/0 (socket says 172.21.15.82:42478) 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 -- 172.21.15.82:0/570230825 learned_addr learned my addr 172.21.15.82:0/570230825 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb98259c700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb984073020 0x7fb984197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb982d9d700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984107af0 0x7fb9841990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 -- 172.21.15.82:0/570230825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984107af0 msgr2=0x7fb9841990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984107af0 0x7fb9841990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 -- 172.21.15.82:0/570230825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb984073020 msgr2=0x7fb984197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb984073020 0x7fb984197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb981d9b700 1 -- 172.21.15.82:0/570230825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9740095f0 con 0x7fb984074600 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.386+0000 7fb982d9d700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984107af0 0x7fb9841990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.387+0000 7fb981d9b700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb9841982f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d980 tx=0x7fb97800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.387+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978004220 con 0x7fb984074600 2023-07-14T19:28:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.387+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb984199db0 con 0x7fb984074600 2023-07-14T19:28:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.387+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb978007640 con 0x7fb984074600 2023-07-14T19:28:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.387+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978005610 con 0x7fb984074600 2023-07-14T19:28:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.387+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9841a0540 con 0x7fb984074600 2023-07-14T19:28:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.388+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984106230 con 0x7fb984074600 2023-07-14T19:28:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.389+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7fb978004750 con 0x7fb984074600 2023-07-14T19:28:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.390+0000 7fb9737fe700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 0x7fb96c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.390+0000 7fb98259c700 1 -- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 msgr2=0x7fb96c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.390+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7fb9780cc820 con 0x7fb984074600 2023-07-14T19:28:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.390+0000 7fb98259c700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 0x7fb96c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:28:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.394+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb978095f90 con 0x7fb984074600 2023-07-14T19:28:27.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.137+0000 7fd3007a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-14T19:28:27.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.224+0000 7fd3007a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:28:27.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.317+0000 7fd3007a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:28:27.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.228+0000 7fad3b7ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:28:27.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.332+0000 7fad3b7ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:28:27.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.225+0000 7f89d5d64200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-14T19:28:27.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.318+0000 7f89d5d64200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-14T19:28:27.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.588+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb98402cd40 con 0x7fb984074600 2023-07-14T19:28:27.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.591+0000 7fb98259c700 1 -- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 msgr2=0x7fb96c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:27.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.591+0000 7fb98259c700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 0x7fb96c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:28:27.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.592+0000 7fb9737fe700 1 -- 172.21.15.82:0/570230825 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb9780160c0 con 0x7fb984074600 2023-07-14T19:28:27.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 msgr2=0x7fb96c0a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:28:27.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 0x7fb96c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 msgr2=0x7fb9841982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb9841982f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb97800d980 tx=0x7fb97800dc90 comp rx=0 tx=0).stop 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 shutdown_connections 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb984107af0 0x7fb9841990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb984073020 0x7fb984197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7fb96c0a11a0 0x7fb96c0a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.594+0000 7fb988ed6700 1 --2- 172.21.15.82:0/570230825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb984074600 0x7fb9841982f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.595+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 >> 172.21.15.82:0/570230825 conn(0x7fb9840fb260 msgr2=0x7fb9840fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.595+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 shutdown_connections 2023-07-14T19:28:27.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.595+0000 7fb988ed6700 1 -- 172.21.15.82:0/570230825 wait complete. 2023-07-14T19:28:27.608 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-14T19:28:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 -- 172.21.15.82:0/3014091458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 msgr2=0x7f9108107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 --2- 172.21.15.82:0/3014091458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108107d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9100009ec0 tx=0x7f910000b330 comp rx=0 tx=0).stop 2023-07-14T19:28:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 -- 172.21.15.82:0/3014091458 shutdown_connections 2023-07-14T19:28:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 --2- 172.21.15.82:0/3014091458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108107d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 --2- 172.21.15.82:0/3014091458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9108100750 0x7f9108102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 --2- 172.21.15.82:0/3014091458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9108103070 0x7f9108105450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 -- 172.21.15.82:0/3014091458 >> 172.21.15.82:0/3014091458 conn(0x7f91080fa130 msgr2=0x7f91080fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 -- 172.21.15.82:0/3014091458 shutdown_connections 2023-07-14T19:28:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.733+0000 7f910ca66700 1 -- 172.21.15.82:0/3014091458 wait complete. 2023-07-14T19:28:27.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 Processor -- start 2023-07-14T19:28:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 -- start start 2023-07-14T19:28:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9108100750 0x7f9108195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9108103070 0x7f9108195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91081974b0 con 0x7f9108105990 2023-07-14T19:28:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9108197620 con 0x7f9108100750 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910ca66700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91081977e0 con 0x7f9108103070 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42546/0 (socket says 172.21.15.82:42546) 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 -- 172.21.15.82:0/2630594831 learned_addr learned my addr 172.21.15.82:0/2630594831 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9108100750 0x7f9108195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9105d9b700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9108103070 0x7f9108195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9108103070 msgr2=0x7f9108195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9108103070 0x7f9108195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9108100750 msgr2=0x7f9108195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9108100750 0x7f9108195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.734+0000 7f9106d9d700 1 -- 172.21.15.82:0/2630594831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100005e70 con 0x7f9108105990 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.735+0000 7f9106d9d700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108196e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9100009fd0 tx=0x7f9100009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.735+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910000e040 con 0x7f9108105990 2023-07-14T19:28:27.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.735+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f910001d940 con 0x7f9108105990 2023-07-14T19:28:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.735+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9108197a60 con 0x7f9108105990 2023-07-14T19:28:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.735+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910001b9b0 con 0x7f9108105990 2023-07-14T19:28:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.735+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910819e280 con 0x7f9108105990 2023-07-14T19:28:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.736+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91080fbca0 con 0x7f9108105990 2023-07-14T19:28:27.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.737+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f910001dab0 con 0x7f9108105990 2023-07-14T19:28:27.743 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.489+0000 7f89d5d64200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:28:27.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.738+0000 7f90f77fe700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.738+0000 7f910659c700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 msgr2=0x7f90f00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.738+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8124+0+0 (secure 0 0 0) 0x7f91000cec00 con 0x7f9108105990 2023-07-14T19:28:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.738+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-14T19:28:27.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.742+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f91000984a0 con 0x7f9108105990 2023-07-14T19:28:27.796 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.487+0000 7fd3007a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:28:27.835 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.510+0000 7fad3b7ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-14T19:28:27.835 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.833+0000 7fad3b7ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:28:27.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.939+0000 7f910659c700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 msgr2=0x7f90f00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:27.940 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.939+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-14T19:28:27.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:27.966+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 --> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9108060a80 con 0x7f90f00a11a0 2023-07-14T19:28:28.105 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.795+0000 7fd3007a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:28:28.106 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:27.883+0000 7fd3007a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:28:28.106 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:28 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:28.011+0000 7fd3007a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:28:28.106 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: scrub queued for path: / 2023-07-14T19:28:28.106 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: scrub summary: idle+waiting paths [/] 2023-07-14T19:28:28.107 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: scrub summary: active paths [/] 2023-07-14T19:28:28.107 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: scrub summary: idle+waiting paths [/] 2023-07-14T19:28:28.107 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1757288499' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-14T19:28:28.107 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-07-14T19:28:28.107 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3694634635' entity='client.admin' 2023-07-14T19:28:28.107 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:27 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/570230825' entity='client.admin' 2023-07-14T19:28:28.112 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.798+0000 7f89d5d64200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-14T19:28:28.112 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:27.887+0000 7f89d5d64200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:28:28.112 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:28 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:28.016+0000 7f89d5d64200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:28:28.112 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: scrub queued for path: / 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: scrub summary: idle+waiting paths [/] 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: scrub summary: active paths [/] 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: scrub summary: idle+waiting paths [/] 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1757288499' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3694634635' entity='client.admin' 2023-07-14T19:28:28.113 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:27 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/570230825' entity='client.admin' 2023-07-14T19:28:28.148 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:27.924+0000 7fad3b7ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-14T19:28:28.148 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:28.053+0000 7fad3b7ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-14T19:28:28.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: scrub queued for path: / 2023-07-14T19:28:28.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: scrub summary: idle+waiting paths [/] 2023-07-14T19:28:28.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: scrub summary: active paths [/] 2023-07-14T19:28:28.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: scrub summary: idle+waiting paths [/] 2023-07-14T19:28:28.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1757288499' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-14T19:28:28.149 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-07-14T19:28:28.150 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3694634635' entity='client.admin' 2023-07-14T19:28:28.150 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:27 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/570230825' entity='client.admin' 2023-07-14T19:28:28.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:28.340+0000 7f910659c700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 msgr2=0x7f90f00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:28.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:28.340+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-14T19:28:28.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:28 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:28.104+0000 7fd3007a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:28:28.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:28.146+0000 7fad3b7ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:28:28.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:28 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:28.110+0000 7f89d5d64200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-14T19:28:28.933 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:28 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:28.523+0000 7fd3007a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:28:28.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:28.564+0000 7fad3b7ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:28:28.991 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:28 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:28.532+0000 7f89d5d64200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-14T19:28:29.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:29.141+0000 7f910659c700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 msgr2=0x7f90f00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:29.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:29.141+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-14T19:28:29.433 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:29 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:29.083+0000 7fd3007a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:28:29.434 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:29 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:29.254+0000 7fd3007a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:28:29.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:29 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:29.122+0000 7fad3b7ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:28:29.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:29 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:29.291+0000 7fad3b7ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:28:29.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:29 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:29.097+0000 7f89d5d64200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-14T19:28:29.492 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:29 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:29.267+0000 7f89d5d64200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-14T19:28:30.276 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:30.017+0000 7fd3007a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:28:30.277 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:30.107+0000 7fd3007a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:28:30.294 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:30.032+0000 7f89d5d64200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:28:30.294 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:30.122+0000 7f89d5d64200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:28:30.321 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:30.056+0000 7fad3b7ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:28:30.321 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:30.146+0000 7fad3b7ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-14T19:28:30.550 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:30.275+0000 7fd3007a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:28:30.569 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:30.292+0000 7f89d5d64200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:28:30.601 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:30.319+0000 7fad3b7ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-14T19:28:30.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:30.742+0000 7f910659c700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 msgr2=0x7f90f00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.82:6800/504215117 2023-07-14T19:28:30.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:30.742+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-14T19:28:30.924 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:30.549+0000 7fd3007a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:28:30.925 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:30.777+0000 7fd3007a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:28:30.944 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:30.567+0000 7f89d5d64200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:28:30.944 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:30.796+0000 7f89d5d64200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:28:30.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:30.599+0000 7fad3b7ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-14T19:28:30.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:30.826+0000 7fad3b7ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-14T19:28:31.183 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:30.923+0000 7fd3007a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:28:31.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:30 smithi133 ceph-mon[128894]: scrub summary: idle 2023-07-14T19:28:31.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-mon[129184]: scrub summary: idle 2023-07-14T19:28:31.206 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:30.942+0000 7f89d5d64200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:28:31.240 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:30.974+0000 7fad3b7ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-14T19:28:31.241 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:30 smithi082 ceph-mon[127461]: scrub summary: idle 2023-07-14T19:28:31.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:31.204+0000 7f89d5d64200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:28:31.683 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:31 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:31.182+0000 7fd3007a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:28:31.714 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:31.239+0000 7fad3b7ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-14T19:28:32.183 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:31 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:31.879+0000 7fd3007a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:28:32.183 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:32.008+0000 7fd3007a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:28:32.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:31.943+0000 7fad3b7ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:28:32.214 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:32.074+0000 7fad3b7ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:28:32.226 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:31.905+0000 7f89d5d64200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-14T19:28:32.226 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:32.035+0000 7f89d5d64200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-14T19:28:32.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:32.224+0000 7f89d5d64200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:28:32.491 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:32.337+0000 7f89d5d64200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:28:32.524 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:32.196+0000 7fd3007a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:28:32.525 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:32.307+0000 7fd3007a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:28:32.604 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:32.267+0000 7fad3b7ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-14T19:28:32.604 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:32.382+0000 7fad3b7ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-14T19:28:32.933 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:32.522+0000 7fd3007a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:28:32.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:32.602+0000 7fad3b7ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:28:32.991 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:32.554+0000 7f89d5d64200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-14T19:28:33.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:33.490+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 20) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f9100098870 con 0x7f9108105990 2023-07-14T19:28:33.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:33.490+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 msgr2=0x7f90f00a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:28:33.492 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:33.490+0000 7f90f77fe700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/504215117,v1:172.21.15.82:6801/504215117] conn(0x7f90f00a11a0 0x7f90f00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:33.683 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y[130343]: 2023-07-14T19:28:33.379+0000 7fd3007a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Standby manager daemon y restarted 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Standby manager daemon y started 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:28:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Standby manager daemon z restarted 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Standby manager daemon z started 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Active manager daemon x restarted 2023-07-14T19:28:33.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Activating manager daemon x 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: mgrmap e20: x(active, starting, since 0.0118478s), standbys: y, z 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T19:28:33.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:28:33.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:28:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:28:33.689 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: Manager daemon x is now available 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:33.690 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:33 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:33.707 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z[131089]: 2023-07-14T19:28:33.416+0000 7f89d5d64200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:28:33.708 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Standby manager daemon y restarted 2023-07-14T19:28:33.708 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Standby manager daemon y started 2023-07-14T19:28:33.708 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T19:28:33.708 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:28:33.708 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T19:28:33.708 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Standby manager daemon z restarted 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Standby manager daemon z started 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:28:33.709 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Active manager daemon x restarted 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Activating manager daemon x 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: mgrmap e20: x(active, starting, since 0.0118478s), standbys: y, z 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T19:28:33.710 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:28:33.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:28:33.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:28:33.713 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:28:33.713 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:28:33.713 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:28:33.713 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:28:33.713 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:28:33.713 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T19:28:33.714 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: Manager daemon x is now available 2023-07-14T19:28:33.715 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:33.715 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:33 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:33.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.467+0000 7fad3b7ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-14T19:28:33.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.668+0000 7fac97c66700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.964 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.668+0000 7fac97c66700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.668+0000 7fac97c66700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.668+0000 7fac97c66700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.668+0000 7fac97c66700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.671+0000 7fac99c6a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.671+0000 7fac99c6a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.671+0000 7fac99c6a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.965 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.671+0000 7fac99c6a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.966 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:28:33.671+0000 7fac99c6a700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:28:33.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Standby manager daemon y restarted 2023-07-14T19:28:33.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Standby manager daemon y started 2023-07-14T19:28:33.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-14T19:28:33.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.133:0/4212527068' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Standby manager daemon z restarted 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Standby manager daemon z started 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-14T19:28:33.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.? 172.21.15.161:0/72745517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Active manager daemon x restarted 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Activating manager daemon x 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: osdmap e84: 12 total, 12 up, 12 in 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: mgrmap e20: x(active, starting, since 0.0118478s), standbys: y, z 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-14T19:28:33.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:28:33.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-14T19:28:33.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-14T19:28:33.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-14T19:28:33.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: Manager daemon x is now available 2023-07-14T19:28:33.973 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:33.973 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:33 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:34.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.493+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mon.0 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f910000fd10 con 0x7f9108105990 2023-07-14T19:28:34.500 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.494+0000 7f90f77fe700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f90f0069750 0x7f90f0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:34.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.494+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 --> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9108060a80 con 0x7f90f0069750 2023-07-14T19:28:34.501 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.499+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f90f0069750 0x7f90f0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:34.502 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.501+0000 7f910659c700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f90f0069750 0x7f90f0069b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91080684f0 tx=0x7f90f8009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:34.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.558+0000 7f90f77fe700 1 -- 172.21.15.82:0/2630594831 <== mgr.15414 v2:172.21.15.82:6800/2084410629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9108060a80 con 0x7f90f0069750 2023-07-14T19:28:34.560 INFO:teuthology.orchestra.run.smithi082.stdout:Schedule set for path / 2023-07-14T19:28:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f90f0069750 msgr2=0x7f90f0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:34.561 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f90f0069750 0x7f90f0069b20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91080684f0 tx=0x7f90f8009260 comp rx=0 tx=0).stop 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 msgr2=0x7f9108196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 --2- 172.21.15.82:0/2630594831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9108105990 0x7f9108196e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9100009fd0 tx=0x7f9100009510 comp rx=0 tx=0).stop 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f9106d9d700 1 -- 172.21.15.82:0/2630594831 reap_dead start 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 shutdown_connections 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 >> 172.21.15.82:0/2630594831 conn(0x7f91080fa130 msgr2=0x7f91081013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 shutdown_connections 2023-07-14T19:28:34.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.560+0000 7f910ca66700 1 -- 172.21.15.82:0/2630594831 wait complete. 2023-07-14T19:28:34.574 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-14T19:28:34.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 -- 172.21.15.82:0/2121275940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab14100770 msgr2=0x7fab14107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:34.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 --2- 172.21.15.82:0/2121275940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab14100770 0x7fab14107010 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab04009bc0 tx=0x7fab04009ed0 comp rx=0 tx=0).stop 2023-07-14T19:28:34.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 -- 172.21.15.82:0/2121275940 shutdown_connections 2023-07-14T19:28:34.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 --2- 172.21.15.82:0/2121275940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab14100770 0x7fab14107010 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:34.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 --2- 172.21.15.82:0/2121275940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab140feb30 0x7fab140fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:34.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 --2- 172.21.15.82:0/2121275940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab140ffdc0 0x7fab14100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:34.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 -- 172.21.15.82:0/2121275940 >> 172.21.15.82:0/2121275940 conn(0x7fab140fa130 msgr2=0x7fab140fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:34.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 -- 172.21.15.82:0/2121275940 shutdown_connections 2023-07-14T19:28:34.708 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.705+0000 7fab18d5a700 1 -- 172.21.15.82:0/2121275940 wait complete. 2023-07-14T19:28:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 Processor -- start 2023-07-14T19:28:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 -- start start 2023-07-14T19:28:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab140feb30 0x7fab141985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 0x7fab1419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab14100770 0x7fab1419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1419d640 con 0x7fab140ffdc0 2023-07-14T19:28:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1419d7b0 con 0x7fab140feb30 2023-07-14T19:28:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab18d5a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1419d970 con 0x7fab14100770 2023-07-14T19:28:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 0x7fab1419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 0x7fab1419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42836/0 (socket says 172.21.15.82:42836) 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 -- 172.21.15.82:0/3906347324 learned_addr learned my addr 172.21.15.82:0/3906347324 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab1259c700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab140feb30 0x7fab141985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab12d9d700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab14100770 0x7fab1419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 -- 172.21.15.82:0/3906347324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab14100770 msgr2=0x7fab1419d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab14100770 0x7fab1419d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:34.712 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 -- 172.21.15.82:0/3906347324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab140feb30 msgr2=0x7fab141985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:34.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab140feb30 0x7fab141985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:34.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.706+0000 7fab11d9b700 1 -- 172.21.15.82:0/3906347324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04009820 con 0x7fab140ffdc0 2023-07-14T19:28:34.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.707+0000 7fab11d9b700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 0x7fab1419aaf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faafc00d980 tx=0x7faafc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:34.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.707+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc004220 con 0x7fab140ffdc0 2023-07-14T19:28:34.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.707+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faafc007640 con 0x7fab140ffdc0 2023-07-14T19:28:34.713 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.707+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1419dc50 con 0x7fab140ffdc0 2023-07-14T19:28:34.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.707+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc005690 con 0x7fab140ffdc0 2023-07-14T19:28:34.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.707+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1419e160 con 0x7fab140ffdc0 2023-07-14T19:28:34.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.708+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab14105a60 con 0x7fab140ffdc0 2023-07-14T19:28:34.714 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.709+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7faafc004750 con 0x7fab140ffdc0 2023-07-14T19:28:34.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.710+0000 7fab0b7fe700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fab000a0f50 0x7fab000a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:34.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.710+0000 7fab1259c700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fab000a0f50 0x7fab000a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:34.715 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.710+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7faafc0cc2b0 con 0x7fab140ffdc0 2023-07-14T19:28:34.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.710+0000 7fab1259c700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fab000a0f50 0x7fab000a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009550 tx=0x7fab0c0091f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:34.716 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.714+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7faafc095c60 con 0x7fab140ffdc0 2023-07-14T19:28:34.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:34 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T19:28:34.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:34 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T19:28:34.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:34 smithi133 ceph-mon[128894]: [14/Jul/2023:19:28:33] ENGINE Bus STARTING 2023-07-14T19:28:34.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:34 smithi133 ceph-mon[128894]: mgrmap e21: x(active, since 1.01724s), standbys: y, z 2023-07-14T19:28:34.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.948+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 --> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fab14060a80 con 0x7fab000a0f50 2023-07-14T19:28:34.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:34 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T19:28:34.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:34 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T19:28:34.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:34 smithi082 ceph-mon[127461]: [14/Jul/2023:19:28:33] ENGINE Bus STARTING 2023-07-14T19:28:34.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:34 smithi082 ceph-mon[127461]: mgrmap e21: x(active, since 1.01724s), standbys: y, z 2023-07-14T19:28:34.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:34 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-14T19:28:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:34 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-14T19:28:34.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:34 smithi161 ceph-mon[129184]: [14/Jul/2023:19:28:33] ENGINE Bus STARTING 2023-07-14T19:28:34.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:34 smithi161 ceph-mon[129184]: mgrmap e21: x(active, since 1.01724s), standbys: y, z 2023-07-14T19:28:35.001 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:34.999+0000 7fab0b7fe700 1 -- 172.21.15.82:0/3906347324 <== mgr.15414 v2:172.21.15.82:6800/2084410629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fab14060a80 con 0x7fab000a0f50 2023-07-14T19:28:35.001 INFO:teuthology.orchestra.run.smithi082.stdout:Retention added to path / 2023-07-14T19:28:35.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fab000a0f50 msgr2=0x7fab000a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fab000a0f50 0x7fab000a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009550 tx=0x7fab0c0091f0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 msgr2=0x7fab1419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 0x7fab1419aaf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faafc00d980 tx=0x7faafc00dc90 comp rx=0 tx=0).stop 2023-07-14T19:28:35.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 shutdown_connections 2023-07-14T19:28:35.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fab000a0f50 0x7fab000a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab14100770 0x7fab1419d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.004 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab140feb30 0x7fab141985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.005 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 --2- 172.21.15.82:0/3906347324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab140ffdc0 0x7fab1419aaf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.001+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 >> 172.21.15.82:0/3906347324 conn(0x7fab140fa130 msgr2=0x7fab140fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.002+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 shutdown_connections 2023-07-14T19:28:35.006 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.002+0000 7fab18d5a700 1 -- 172.21.15.82:0/3906347324 wait complete. 2023-07-14T19:28:35.016 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-14T19:28:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 -- 172.21.15.82:0/3680974137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc158100750 msgr2=0x7fc158107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 --2- 172.21.15.82:0/3680974137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc158100750 0x7fc158107010 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc15000b0a0 tx=0x7fc15000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 -- 172.21.15.82:0/3680974137 shutdown_connections 2023-07-14T19:28:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 --2- 172.21.15.82:0/3680974137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc158100750 0x7fc158107010 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 --2- 172.21.15.82:0/3680974137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1580febd0 0x7fc1580fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 --2- 172.21.15.82:0/3680974137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1580ffdc0 0x7fc158100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 -- 172.21.15.82:0/3680974137 >> 172.21.15.82:0/3680974137 conn(0x7fc1580fa1f0 msgr2=0x7fc1580fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:35.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 -- 172.21.15.82:0/3680974137 shutdown_connections 2023-07-14T19:28:35.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.142+0000 7fc15e977700 1 -- 172.21.15.82:0/3680974137 wait complete. 2023-07-14T19:28:35.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 Processor -- start 2023-07-14T19:28:35.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 -- start start 2023-07-14T19:28:35.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1580febd0 0x7fc158194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 0x7fc158194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc158100750 0x7fc158195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581964b0 con 0x7fc1580ffdc0 2023-07-14T19:28:35.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158196620 con 0x7fc158100750 2023-07-14T19:28:35.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc15e977700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581967e0 con 0x7fc1580febd0 2023-07-14T19:28:35.148 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc1577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 0x7fc158194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc1577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 0x7fc158194fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42850/0 (socket says 172.21.15.82:42850) 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.143+0000 7fc1577fe700 1 -- 172.21.15.82:0/3200249119 learned_addr learned my addr 172.21.15.82:0/3200249119 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc157fff700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1580febd0 0x7fc158194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc15cf14700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc158100750 0x7fc158195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1577fe700 1 -- 172.21.15.82:0/3200249119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1580febd0 msgr2=0x7fc158194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1577fe700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1580febd0 0x7fc158194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1577fe700 1 -- 172.21.15.82:0/3200249119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc158100750 msgr2=0x7fc158195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1577fe700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc158100750 0x7fc158195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1577fe700 1 -- 172.21.15.82:0/3200249119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150009d00 con 0x7fc1580ffdc0 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1577fe700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 0x7fc158194fd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e9a0 tx=0x7fc14800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148009790 con 0x7fc1580ffdc0 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc14800cbd0 con 0x7fc1580ffdc0 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc158196ac0 con 0x7fc1580ffdc0 2023-07-14T19:28:35.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148018680 con 0x7fc1580ffdc0 2023-07-14T19:28:35.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.144+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15819d280 con 0x7fc1580ffdc0 2023-07-14T19:28:35.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.145+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158106120 con 0x7fc1580ffdc0 2023-07-14T19:28:35.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.146+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fc14800cd40 con 0x7fc1580ffdc0 2023-07-14T19:28:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.147+0000 7fc1557fa700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc1400a0fd0 0x7fc1400a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.147+0000 7fc157fff700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc1400a0fd0 0x7fc1400a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.147+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fc148014070 con 0x7fc1580ffdc0 2023-07-14T19:28:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.147+0000 7fc157fff700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc1400a0fd0 0x7fc1400a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ffc20 tx=0x7fc14c009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.150+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc148096440 con 0x7fc1580ffdc0 2023-07-14T19:28:35.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.374+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 --> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc158060a80 con 0x7fc1400a0fd0 2023-07-14T19:28:35.384 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.382+0000 7fc1557fa700 1 -- 172.21.15.82:0/3200249119 <== mgr.15414 v2:172.21.15.82:6800/2084410629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fc158060a80 con 0x7fc1400a0fd0 2023-07-14T19:28:35.384 INFO:teuthology.orchestra.run.smithi082.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-14T00:00:00", "created": "2023-07-14T19:28:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-14T19:28:35.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc1400a0fd0 msgr2=0x7fc1400a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc1400a0fd0 0x7fc1400a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1580ffc20 tx=0x7fc14c009260 comp rx=0 tx=0).stop 2023-07-14T19:28:35.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 msgr2=0x7fc158194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 0x7fc158194fd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e9a0 tx=0x7fc14800ecb0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 shutdown_connections 2023-07-14T19:28:35.386 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc1400a0fd0 0x7fc1400a3480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc158100750 0x7fc158195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc1580febd0 0x7fc158194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 --2- 172.21.15.82:0/3200249119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1580ffdc0 0x7fc158194fd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.384+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 >> 172.21.15.82:0/3200249119 conn(0x7fc1580fa1f0 msgr2=0x7fc158103260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.385+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 shutdown_connections 2023-07-14T19:28:35.387 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.385+0000 7fc15e977700 1 -- 172.21.15.82:0/3200249119 wait complete. 2023-07-14T19:28:35.398 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-14T19:28:35.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 -- 172.21.15.82:0/1241011355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741030b0 msgr2=0x7f3374105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 --2- 172.21.15.82:0/1241011355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741030b0 0x7f3374105490 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3364009990 tx=0x7f3364009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 -- 172.21.15.82:0/1241011355 shutdown_connections 2023-07-14T19:28:35.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 --2- 172.21.15.82:0/1241011355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33741059d0 0x7f3374107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 --2- 172.21.15.82:0/1241011355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3374100790 0x7f3374102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 --2- 172.21.15.82:0/1241011355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741030b0 0x7f3374105490 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 -- 172.21.15.82:0/1241011355 >> 172.21.15.82:0/1241011355 conn(0x7f33740fa130 msgr2=0x7f33740fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 -- 172.21.15.82:0/1241011355 shutdown_connections 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.528+0000 7f337a654700 1 -- 172.21.15.82:0/1241011355 wait complete. 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 Processor -- start 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 -- start start 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3374100790 0x7f3374195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33741030b0 0x7f3374196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 0x7f3374196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33741974e0 con 0x7f33741059d0 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374197650 con 0x7f3374100790 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f337a654700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374197810 con 0x7f33741030b0 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 0x7f3374196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 0x7f3374196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42866/0 (socket says 172.21.15.82:42866) 2023-07-14T19:28:35.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 -- 172.21.15.82:0/877010690 learned_addr learned my addr 172.21.15.82:0/877010690 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3373fff700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3374100790 0x7f3374195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f33737fe700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33741030b0 0x7f3374196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 -- 172.21.15.82:0/877010690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33741030b0 msgr2=0x7f3374196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33741030b0 0x7f3374196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 -- 172.21.15.82:0/877010690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3374100790 msgr2=0x7f3374195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3374100790 0x7f3374195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.529+0000 7f3378bf1700 1 -- 172.21.15.82:0/877010690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33640095f0 con 0x7f33741059d0 2023-07-14T19:28:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.530+0000 7f3378bf1700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 0x7f3374196e80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f336c00ef30 tx=0x7f336c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.530+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c018070 con 0x7f33741059d0 2023-07-14T19:28:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.530+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336c00f040 con 0x7f33741059d0 2023-07-14T19:28:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.530+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c007be0 con 0x7f33741059d0 2023-07-14T19:28:35.537 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.530+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3374197a90 con 0x7f33741059d0 2023-07-14T19:28:35.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.530+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337419e2b0 con 0x7f33741059d0 2023-07-14T19:28:35.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.531+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33740fbca0 con 0x7f33741059d0 2023-07-14T19:28:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.532+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f336c007d40 con 0x7f33741059d0 2023-07-14T19:28:35.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.533+0000 7f33717fa700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f335c0a1010 0x7f335c0a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:35.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.533+0000 7f3373fff700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f335c0a1010 0x7f335c0a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:35.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.533+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f336c0ccf80 con 0x7f33741059d0 2023-07-14T19:28:35.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.533+0000 7f3373fff700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f335c0a1010 0x7f335c0a34c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3368005950 tx=0x7f336800b410 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:35.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.536+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f336c0969c0 con 0x7f33741059d0 2023-07-14T19:28:35.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:35 smithi082 ceph-mon[127461]: [14/Jul/2023:19:28:34] ENGINE Serving on https://172.21.15.82:7150 2023-07-14T19:28:35.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:35 smithi082 ceph-mon[127461]: [14/Jul/2023:19:28:34] ENGINE Serving on http://172.21.15.82:8765 2023-07-14T19:28:35.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:35 smithi082 ceph-mon[127461]: [14/Jul/2023:19:28:34] ENGINE Bus STARTED 2023-07-14T19:28:35.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:35 smithi082 ceph-mon[127461]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:35.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:35 smithi082 ceph-mon[127461]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.768+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 --> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3374060a80 con 0x7f335c0a1010 2023-07-14T19:28:35.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.775+0000 7f33717fa700 1 -- 172.21.15.82:0/877010690 <== mgr.15414 v2:172.21.15.82:6800/2084410629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3374060a80 con 0x7f335c0a1010 2023-07-14T19:28:35.778 INFO:teuthology.orchestra.run.smithi082.stdout:/ 1M 6M3h 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f335c0a1010 msgr2=0x7f335c0a34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f335c0a1010 0x7f335c0a34c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3368005950 tx=0x7f336800b410 comp rx=0 tx=0).stop 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 msgr2=0x7f3374196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 0x7f3374196e80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f336c00ef30 tx=0x7f336c00c3a0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 shutdown_connections 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f335c0a1010 0x7f335c0a34c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33741059d0 0x7f3374196e80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3374100790 0x7f3374195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 --2- 172.21.15.82:0/877010690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33741030b0 0x7f3374196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 >> 172.21.15.82:0/877010690 conn(0x7f33740fa130 msgr2=0x7f33741013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 shutdown_connections 2023-07-14T19:28:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:35.777+0000 7f337a654700 1 -- 172.21.15.82:0/877010690 wait complete. 2023-07-14T19:28:35.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-14T19:28:35.821 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-14T19:28:35.832 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-14T19:28:35.855 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:28:35.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-14T19:28:35.863 INFO:teuthology.run_tasks:Running task workunit... 2023-07-14T19:28:35.876 INFO:tasks.workunit:Pulling workunits from ref 81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:28:35.877 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-14T19:28:35.877 INFO:tasks.workunit:timeout=3h 2023-07-14T19:28:35.878 INFO:tasks.workunit:cleanup=True 2023-07-14T19:28:35.878 DEBUG:teuthology.orchestra.run.smithi082:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:35.895 INFO:teuthology.orchestra.run.smithi082.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:35.895 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-14T19:28:35.895 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-07-14T19:28:35.895 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-14T19:28:35.896 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-14T19:28:35.896 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-07-14 19:27:03.685778402 +0000 2023-07-14T19:28:35.896 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-07-14 19:27:03.685778402 +0000 2023-07-14T19:28:35.896 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-07-14 19:28:18.924875347 +0000 2023-07-14T19:28:35.896 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2023-07-14 19:27:03.685778402 +0000 2023-07-14T19:28:35.897 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-14T19:28:35.897 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-14T19:28:35.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:35 smithi133 ceph-mon[128894]: [14/Jul/2023:19:28:34] ENGINE Serving on https://172.21.15.82:7150 2023-07-14T19:28:35.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:35 smithi133 ceph-mon[128894]: [14/Jul/2023:19:28:34] ENGINE Serving on http://172.21.15.82:8765 2023-07-14T19:28:35.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:35 smithi133 ceph-mon[128894]: [14/Jul/2023:19:28:34] ENGINE Bus STARTED 2023-07-14T19:28:35.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:35 smithi133 ceph-mon[128894]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:35.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:35 smithi133 ceph-mon[128894]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:35.976 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 81c9387d2e7576da065747ec9c16d1c7d69016ec 2023-07-14T19:28:35.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:35 smithi161 ceph-mon[129184]: [14/Jul/2023:19:28:34] ENGINE Serving on https://172.21.15.82:7150 2023-07-14T19:28:35.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:35 smithi161 ceph-mon[129184]: [14/Jul/2023:19:28:34] ENGINE Serving on http://172.21.15.82:8765 2023-07-14T19:28:35.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:35 smithi161 ceph-mon[129184]: [14/Jul/2023:19:28:34] ENGINE Bus STARTED 2023-07-14T19:28:35.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:35 smithi161 ceph-mon[129184]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:35.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:35 smithi161 ceph-mon[129184]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:36.033 INFO:tasks.workunit.client.0.smithi082.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-14T19:28:36.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:36.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:36.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:36.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:36 smithi133 ceph-mon[128894]: mgrmap e22: x(active, since 3s), standbys: y, z 2023-07-14T19:28:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:36.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:36.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:36 smithi082 ceph-mon[127461]: mgrmap e22: x(active, since 3s), standbys: y, z 2023-07-14T19:28:36.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:28:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:36.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:36 smithi161 ceph-mon[129184]: mgrmap e22: x(active, since 3s), standbys: y, z 2023-07-14T19:28:37.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:37.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:37 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:37.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:37.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:37 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:37.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:37.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:37 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi133 to 196.2M 2023-07-14T19:28:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Unable to set osd_memory_target on smithi133 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T19:28:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T19:28:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T19:28:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-07-14T19:28:38.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi133 to 196.2M 2023-07-14T19:28:38.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Unable to set osd_memory_target on smithi133 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T19:28:38.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T19:28:38.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T19:28:38.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-07-14T19:28:38.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-07-14T19:28:38.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:28:38.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:28:38.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:28:38.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:38.966 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi133 to 196.2M 2023-07-14T19:28:38.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Unable to set osd_memory_target on smithi133 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T19:28:38.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T19:28:38.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T19:28:38.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-07-14T19:28:38.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-07-14T19:28:38.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Updating smithi082:/etc/ceph/ceph.conf 2023-07-14T19:28:38.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Updating smithi133:/etc/ceph/ceph.conf 2023-07-14T19:28:38.994 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Updating smithi161:/etc/ceph/ceph.conf 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.conf 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:38.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:39.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:39.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:39.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:39.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:39.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:39.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:39.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: Updating smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: Updating smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: Updating smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/config/ceph.client.admin.keyring 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:28:39.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:28:39.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:28:39.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:28:40.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:40 smithi133 ceph-mon[128894]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:40.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:40 smithi082 ceph-mon[127461]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:40 smithi161 ceph-mon[129184]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-14T19:28:42.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:42 smithi133 ceph-mon[128894]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-14T19:28:42.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:42 smithi082 ceph-mon[127461]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-14T19:28:42.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:42 smithi161 ceph-mon[129184]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-07-14T19:28:44.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:44 smithi082 ceph-mon[127461]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-14T19:28:44.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:44 smithi161 ceph-mon[129184]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-14T19:28:45.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:44 smithi133 ceph-mon[128894]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-14T19:28:46.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:46 smithi082 ceph-mon[127461]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-14T19:28:46.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:46 smithi161 ceph-mon[129184]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-14T19:28:47.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:46 smithi133 ceph-mon[128894]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-14T19:28:48.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:48 smithi082 ceph-mon[127461]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T19:28:48.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:48 smithi161 ceph-mon[129184]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T19:28:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3886758167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3380ffbe0 msgr2=0x7fc33810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3886758167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3380ffbe0 0x7fc33810b970 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b0d0 tx=0x7fc33000b3e0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3886758167 shutdown_connections 2023-07-14T19:28:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3886758167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3380ffbe0 0x7fc33810b970 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3886758167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc338103ae0 0x7fc338103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3886758167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380ff230 0x7fc3380ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3886758167 >> 172.21.15.82:0/3886758167 conn(0x7fc3380760c0 msgr2=0x7fc3380764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3886758167 shutdown_connections 2023-07-14T19:28:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.026+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3886758167 wait complete. 2023-07-14T19:28:49.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 Processor -- start 2023-07-14T19:28:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 -- start start 2023-07-14T19:28:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3380ff230 0x7fc3381009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380ffbe0 0x7fc338100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 0x7fc3381a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381a44b0 con 0x7fc338103ae0 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381a45f0 con 0x7fc3380ffbe0 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc33ddd3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381a4760 con 0x7fc3380ff230 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc337fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 0x7fc3381a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc337fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 0x7fc3381a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42880/0 (socket says 172.21.15.82:42880) 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.027+0000 7fc337fff700 1 -- 172.21.15.82:0/3728314936 learned_addr learned my addr 172.21.15.82:0/3728314936 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:49.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc336ffd700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380ffbe0 0x7fc338100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc3377fe700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3380ff230 0x7fc3381009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc337fff700 1 -- 172.21.15.82:0/3728314936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3380ff230 msgr2=0x7fc3381009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc337fff700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3380ff230 0x7fc3381009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc337fff700 1 -- 172.21.15.82:0/3728314936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380ffbe0 msgr2=0x7fc338100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc337fff700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380ffbe0 0x7fc338100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc337fff700 1 -- 172.21.15.82:0/3728314936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc330009d00 con 0x7fc338103ae0 2023-07-14T19:28:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc337fff700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 0x7fc3381a3f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc330007870 tx=0x7fc3300078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc334ff9700 1 -- 172.21.15.82:0/3728314936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33000e040 con 0x7fc338103ae0 2023-07-14T19:28:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3381a49e0 con 0x7fc338103ae0 2023-07-14T19:28:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc334ff9700 1 -- 172.21.15.82:0/3728314936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc330005070 con 0x7fc338103ae0 2023-07-14T19:28:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3381a4e10 con 0x7fc338103ae0 2023-07-14T19:28:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.028+0000 7fc334ff9700 1 -- 172.21.15.82:0/3728314936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33001b660 con 0x7fc338103ae0 2023-07-14T19:28:49.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.030+0000 7fc334ff9700 1 -- 172.21.15.82:0/3728314936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fc3300051e0 con 0x7fc338103ae0 2023-07-14T19:28:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.031+0000 7fc334ff9700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc3200a1350 0x7fc3200a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.031+0000 7fc3377fe700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc3200a1350 0x7fc3200a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.031+0000 7fc334ff9700 1 -- 172.21.15.82:0/3728314936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fc3300cdc70 con 0x7fc338103ae0 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.031+0000 7fc3377fe700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc3200a1350 0x7fc3200a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009bf0 tx=0x7fc32c009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 Processor -- start 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 -- start start 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 0x556a56e1f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1ff00 0x556a56e19f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56d1ec50 con 0x556a56c756b0 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56d719e0 con 0x556a56e1fa90 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56d6ae70 con 0x556a56e1ff00 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc3377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 0x556a56e1f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc3377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 0x556a56e1f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42884/0 (socket says 172.21.15.82:42884) 2023-07-14T19:28:49.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc3377fe700 1 -- 172.21.15.82:0/2409393760 learned_addr learned my addr 172.21.15.82:0/2409393760 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc336ffd700 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.032+0000 7fc337fff700 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1ff00 0x556a56e19f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc3377fe700 1 -- 172.21.15.82:0/2409393760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1ff00 msgr2=0x556a56e19f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1ff00 0x556a56e19f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc3377fe700 1 -- 172.21.15.82:0/2409393760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1fa90 msgr2=0x556a56e199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc3377fe700 1 -- 172.21.15.82:0/2409393760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a56b98d80 con 0x556a56c756b0 2023-07-14T19:28:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 0x556a56e1f550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc32c011e00 tx=0x7fc32c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c232aadb90170da3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc31dffb700 1 -- 172.21.15.82:0/2409393760 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c012bc0 con 0x556a56c756b0 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc31dffb700 1 -- 172.21.15.82:0/2409393760 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc32c016070 con 0x556a56c756b0 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.033+0000 7fc31dffb700 1 -- 172.21.15.82:0/2409393760 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c003cd0 con 0x556a56c756b0 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2409393760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 msgr2=0x556a56e1f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 0x556a56e1f550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc32c011e00 tx=0x7fc32c014000 comp rx=0 tx=0).stop 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2409393760 shutdown_connections 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1ff00 0x556a56e19f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56c756b0 0x556a56e1f550 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2409393760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2409393760 >> 172.21.15.82:0/2409393760 conn(0x556a56e0f3d0 msgr2=0x556a56e14530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2409393760 shutdown_connections 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2409393760 wait complete. 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- start start 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 Processor -- start 2023-07-14T19:28:49.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56c756b0 0x556a56e28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1fa90 0x556a56e29310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 0x556a56e23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.034+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56d0aa60 con 0x556a56e1ff00 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56d0a760 con 0x556a56c756b0 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56bee8c0 con 0x556a56e1fa90 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc349f6bb80 10 client.0 _ll_register_callbacks cb 0x556a56d08700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:49.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 0x556a56e23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 0x556a56e23090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42894/0 (socket says 172.21.15.82:42894) 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 -- 172.21.15.82:0/2116179207 learned_addr learned my addr 172.21.15.82:0/2116179207 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56c756b0 0x556a56e28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc336ffd700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1fa90 0x556a56e29310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 -- 172.21.15.82:0/2116179207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1fa90 msgr2=0x556a56e29310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1fa90 0x556a56e29310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 -- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56c756b0 msgr2=0x556a56e28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56c756b0 0x556a56e28dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 -- 172.21.15.82:0/2116179207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c011ae0 con 0x556a56e1ff00 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc337fff700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 0x556a56e23090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc330009cd0 tx=0x7fc33009a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f4f3052f2d8f89 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc316ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc316ffd700 1 -- 172.21.15.82:0/2116179207 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3300b3420 con 0x556a56e1ff00 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc316ffd700 1 -- 172.21.15.82:0/2116179207 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fc3300a3050 con 0x556a56e1ff00 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a56cc4490 con 0x556a56e1ff00 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc316ffd700 1 -- 172.21.15.82:0/2116179207 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc33009aef0 con 0x556a56e1ff00 2023-07-14T19:28:49.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc349f6bb80 10 client.15516 fetch_fsmap learned FSMap version 23 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.035+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556a56b81c00 con 0x556a56e1ff00 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc316ffd700 1 -- 172.21.15.82:0/2116179207 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a56cc4490 con 0x556a56e1ff00 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc316ffd700 1 -- 172.21.15.82:0/2116179207 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc3300a2bc0 con 0x556a56e1ff00 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc349f6bb80 10 client.15516 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc349f6bb80 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc349f6bb80 20 client.15516 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e33020 0x556a56e35400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc349f6bb80 4 client.15516 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a56de8d40 con 0x556a56e33020 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e33020 0x556a56e35400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc316ffd700 10 client.15516 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.036+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e33020 0x556a56e35400 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56e12810549765fc server_cookie=8ade3137d76e8b1 in_seq=0 out_seq=0 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc316ffd700 1 -- 172.21.15.82:0/2116179207 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556a56de8d40 con 0x556a56e33020 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc316ffd700 10 client.15516 handle_command_reply: tid=0 2023-07-14T19:28:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 client.15516 shutdown 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 msgr2=0x556a56e23090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 0x556a56e23090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc330009cd0 tx=0x7fc33009a670 comp rx=0 tx=0).stop 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 shutdown_connections 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1ff00 0x556a56e23090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56c756b0 0x556a56e28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e33020 0x556a56e35400 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2116179207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56e1fa90 0x556a56e29310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc316ffd700 0 client.15516 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 >> 172.21.15.82:0/2116179207 conn(0x556a56e0f3d0 msgr2=0x556a56e13c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 shutdown_connections 2023-07-14T19:28:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.037+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2116179207 wait complete. 2023-07-14T19:28:49.051 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.038+0000 7fc349f6bb80 20 client.15516 trim_cache size 0 max 16384 2023-07-14T19:28:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 Processor -- start 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 -- start start 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e1e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56e24020 con 0x556a56e1fa90 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56e3a200 con 0x556a56e1ff00 2023-07-14T19:28:49.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.052+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56e23520 con 0x556a56c756b0 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42900/0 (socket says 172.21.15.82:42900) 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 -- 172.21.15.82:0/2038370872 learned_addr learned my addr 172.21.15.82:0/2038370872 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc3377fe700 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e1e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc337fff700 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 -- 172.21.15.82:0/2038370872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 msgr2=0x556a56e1e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e1e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 -- 172.21.15.82:0/2038370872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 msgr2=0x556a56e143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 -- 172.21.15.82:0/2038370872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a56bbf1f0 con 0x556a56e1fa90 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc336ffd700 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b8c0 tx=0x7fc32800bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfaf49295b4dc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc31e7fc700 1 -- 172.21.15.82:0/2038370872 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328016070 con 0x556a56e1fa90 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc31e7fc700 1 -- 172.21.15.82:0/2038370872 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc328017040 con 0x556a56e1fa90 2023-07-14T19:28:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.053+0000 7fc31e7fc700 1 -- 172.21.15.82:0/2038370872 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328011e70 con 0x556a56e1fa90 2023-07-14T19:28:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2038370872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 msgr2=0x556a56e199e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b8c0 tx=0x7fc32800bbd0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2038370872 shutdown_connections 2023-07-14T19:28:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e1e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/2038370872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e199e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2038370872 >> 172.21.15.82:0/2038370872 conn(0x556a56e0f3d0 msgr2=0x556a56e1e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2038370872 shutdown_connections 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 -- 172.21.15.82:0/2038370872 wait complete. 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 -- start start 2023-07-14T19:28:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.054+0000 7fc349f6bb80 1 Processor -- start 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e28ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e27740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56e27c80 con 0x556a56e1fa90 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56e27df0 con 0x556a56e1ff00 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a56e27fb0 con 0x556a56c756b0 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 10 client.0 _ll_register_callbacks cb 0x7fc33009a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:49.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e27200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42906/0 (socket says 172.21.15.82:42906) 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 -- 172.21.15.82:0/3647657080 learned_addr learned my addr 172.21.15.82:0/3647657080 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc3377fe700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e28ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc337fff700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e27740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 -- 172.21.15.82:0/3647657080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 msgr2=0x556a56e28ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e28ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 -- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 msgr2=0x556a56e27740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e27740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 -- 172.21.15.82:0/3647657080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3280043a0 con 0x556a56e1fa90 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc336ffd700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e27200 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c010 tx=0x7fc3280116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d5ccd0199f6f59 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc316ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc316ffd700 1 -- 172.21.15.82:0/3647657080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328016070 con 0x556a56e1fa90 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc316ffd700 1 -- 172.21.15.82:0/3647657080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fc3280038a0 con 0x556a56e1fa90 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc32c016070 con 0x556a56e1fa90 2023-07-14T19:28:49.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc316ffd700 1 -- 172.21.15.82:0/3647657080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc328003a60 con 0x556a56e1fa90 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 10 client.15522 fetch_fsmap learned FSMap version 23 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.055+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556a56b97250 con 0x556a56e1fa90 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc316ffd700 1 -- 172.21.15.82:0/3647657080 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc32c016070 con 0x556a56e1fa90 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc316ffd700 1 -- 172.21.15.82:0/3647657080 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc32802ecf0 con 0x556a56e1fa90 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc349f6bb80 10 client.15522 fetch_fsmap finished waiting for FSMap version 23 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc349f6bb80 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc349f6bb80 20 client.15522 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e880c0 0x556a56e8a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc349f6bb80 4 client.15522 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:28:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556a56d63e50 con 0x556a56e880c0 2023-07-14T19:28:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc3377fe700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e880c0 0x556a56e8a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc316ffd700 10 client.15522 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.056+0000 7fc3377fe700 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e880c0 0x556a56e8a4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce7fe355226a455 server_cookie=89d1ca91b14f5b78 in_seq=0 out_seq=0 2023-07-14T19:28:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc316ffd700 1 -- 172.21.15.82:0/3647657080 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556a56d63e50 con 0x556a56e880c0 2023-07-14T19:28:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc316ffd700 10 client.15522 handle_command_reply: tid=0 2023-07-14T19:28:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 client.15522 shutdown 2023-07-14T19:28:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 msgr2=0x556a56e27200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e27200 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc32800c010 tx=0x7fc3280116f0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 shutdown_connections 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a56e1ff00 0x556a56e27740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556a56e880c0 0x556a56e8a4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a56c756b0 0x556a56e28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc316ffd700 0 client.15522 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 --2- 172.21.15.82:0/3647657080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a56e1fa90 0x556a56e27200 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.067+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 >> 172.21.15.82:0/3647657080 conn(0x556a56e0f3d0 msgr2=0x556a56e1e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 shutdown_connections 2023-07-14T19:28:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc349f6bb80 1 -- 172.21.15.82:0/3647657080 wait complete. 2023-07-14T19:28:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc349f6bb80 20 client.15522 trim_cache size 0 max 16384 2023-07-14T19:28:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:28:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:28:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:28:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:28:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc3200a1350 msgr2=0x7fc3200a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc3200a1350 0x7fc3200a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009bf0 tx=0x7fc32c009260 comp rx=0 tx=0).stop 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 msgr2=0x7fc3381a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 0x7fc3381a3f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc330007870 tx=0x7fc3300078a0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 shutdown_connections 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc3200a1350 0x7fc3200a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc338103ae0 0x7fc3381a3f70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc3380ff230 0x7fc3381009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 --2- 172.21.15.82:0/3728314936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3380ffbe0 0x7fc338100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 >> 172.21.15.82:0/3728314936 conn(0x7fc3380760c0 msgr2=0x7fc3380fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 shutdown_connections 2023-07-14T19:28:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:28:49.068+0000 7fc33ddd3700 1 -- 172.21.15.82:0/3728314936 wait complete. 2023-07-14T19:28:49.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:48 smithi133 ceph-mon[128894]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T19:28:50.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:50 smithi082 ceph-mon[127461]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T19:28:50.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:50 smithi161 ceph-mon[129184]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T19:28:51.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:50 smithi133 ceph-mon[128894]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-14T19:28:52.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:52 smithi082 ceph-mon[127461]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 39 op/s 2023-07-14T19:28:52.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:52 smithi161 ceph-mon[129184]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 39 op/s 2023-07-14T19:28:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:52 smithi133 ceph-mon[128894]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 39 op/s 2023-07-14T19:28:54.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:54 smithi082 ceph-mon[127461]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-07-14T19:28:54.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:54 smithi161 ceph-mon[129184]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-07-14T19:28:55.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:54 smithi133 ceph-mon[128894]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-07-14T19:28:56.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T19:28:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 -- 172.21.15.82:0/3437589952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 msgr2=0x7f864810b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 --2- 172.21.15.82:0/3437589952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864810b940 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f864000b0a0 tx=0x7f864000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 -- 172.21.15.82:0/3437589952 shutdown_connections 2023-07-14T19:28:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 --2- 172.21.15.82:0/3437589952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864810b940 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 --2- 172.21.15.82:0/3437589952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8648104b10 0x7f8648104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 --2- 172.21.15.82:0/3437589952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86480feb00 0x7f86480fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 -- 172.21.15.82:0/3437589952 >> 172.21.15.82:0/3437589952 conn(0x7f86480fa5e0 msgr2=0x7f86480fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 -- 172.21.15.82:0/3437589952 shutdown_connections 2023-07-14T19:28:56.770 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.768+0000 7f8650642700 1 -- 172.21.15.82:0/3437589952 wait complete. 2023-07-14T19:28:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 Processor -- start 2023-07-14T19:28:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 -- start start 2023-07-14T19:28:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86480feb00 0x7f864819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648104b10 0x7f8648194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86481950a0 con 0x7f86480ff4b0 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8648195210 con 0x7f86480feb00 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f8650642700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86481953d0 con 0x7f8648104b10 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f864dbdd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f864dbdd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42922/0 (socket says 172.21.15.82:42922) 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f864dbdd700 1 -- 172.21.15.82:0/4125936659 learned_addr learned my addr 172.21.15.82:0/4125936659 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:56.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.769+0000 7f864e3de700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86480feb00 0x7f864819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864ebdf700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648104b10 0x7f8648194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864dbdd700 1 -- 172.21.15.82:0/4125936659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648104b10 msgr2=0x7f8648194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864dbdd700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648104b10 0x7f8648194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864dbdd700 1 -- 172.21.15.82:0/4125936659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86480feb00 msgr2=0x7f864819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864dbdd700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86480feb00 0x7f864819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864dbdd700 1 -- 172.21.15.82:0/4125936659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640009d00 con 0x7f86480ff4b0 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864ebdf700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648104b10 0x7f8648194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:56.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f864dbdd700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864819af00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f863800c9b0 tx=0x7f863800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f863f7fe700 1 -- 172.21.15.82:0/4125936659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638007b00 con 0x7f86480ff4b0 2023-07-14T19:28:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86481956b0 con 0x7f86480ff4b0 2023-07-14T19:28:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f863f7fe700 1 -- 172.21.15.82:0/4125936659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8638007c60 con 0x7f86480ff4b0 2023-07-14T19:28:56.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f863f7fe700 1 -- 172.21.15.82:0/4125936659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638018660 con 0x7f86480ff4b0 2023-07-14T19:28:56.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.770+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86481a19b0 con 0x7f86480ff4b0 2023-07-14T19:28:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.772+0000 7f863f7fe700 1 -- 172.21.15.82:0/4125936659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f863800f450 con 0x7f86480ff4b0 2023-07-14T19:28:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.773+0000 7f863f7fe700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f86340a1350 0x7f86340a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.773+0000 7f864e3de700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f86340a1350 0x7f86340a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.773+0000 7f863f7fe700 1 -- 172.21.15.82:0/4125936659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f86380cce20 con 0x7f86480ff4b0 2023-07-14T19:28:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.773+0000 7f864e3de700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f86340a1350 0x7f86340a3800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f864400abd0 tx=0x7f864400a380 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.774+0000 7f865c7dab80 1 Processor -- start 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.774+0000 7f865c7dab80 1 -- start start 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ff0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f865c7dab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8ef4cb0 con 0x5647a8ff4c30 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f865c7dab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8f47a40 con 0x5647a8ff0790 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f865c7dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8f40ed0 con 0x5647a8fefe80 2023-07-14T19:28:56.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42932/0 (socket says 172.21.15.82:42932) 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 -- 172.21.15.82:0/4254543559 learned_addr learned my addr 172.21.15.82:0/4254543559 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864e3de700 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ff0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864dbdd700 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 -- 172.21.15.82:0/4254543559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 msgr2=0x5647a8ff0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ff0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 -- 172.21.15.82:0/4254543559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 msgr2=0x5647a8ff46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 -- 172.21.15.82:0/4254543559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a8d6ede0 con 0x5647a8ff4c30 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864e3de700 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ff0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f864ebdf700 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff5080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f864000b070 tx=0x7f8640008d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5842bd25adb6d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f863cff9700 1 -- 172.21.15.82:0/4254543559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864000e050 con 0x5647a8ff4c30 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f863cff9700 1 -- 172.21.15.82:0/4254543559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86400092d0 con 0x5647a8ff4c30 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.775+0000 7f863cff9700 1 -- 172.21.15.82:0/4254543559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86400052a0 con 0x5647a8ff4c30 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 -- 172.21.15.82:0/4254543559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 msgr2=0x5647a8ff5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff5080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f864000b070 tx=0x7f8640008d10 comp rx=0 tx=0).stop 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 -- 172.21.15.82:0/4254543559 shutdown_connections 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff5080 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ff0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 --2- 172.21.15.82:0/4254543559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 -- 172.21.15.82:0/4254543559 >> 172.21.15.82:0/4254543559 conn(0x5647a8fe5680 msgr2=0x5647a8feaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 -- 172.21.15.82:0/4254543559 shutdown_connections 2023-07-14T19:28:56.784 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.776+0000 7f865c7dab80 1 -- 172.21.15.82:0/4254543559 wait complete. 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 -- start start 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 Processor -- start 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ffc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8ee0ac0 con 0x5647a8ff4c30 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8ee07c0 con 0x5647a8ff0790 2023-07-14T19:28:56.785 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8dc4920 con 0x5647a8fefe80 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f865c7dab80 10 client.0 _ll_register_callbacks cb 0x5647a8ede760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f864dbdd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f864dbdd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:39284/0 (socket says 172.21.15.82:39284) 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f864dbdd700 1 -- 172.21.15.82:0/3438793929 learned_addr learned my addr 172.21.15.82:0/3438793929 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.777+0000 7f864e3de700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ffc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864dbdd700 1 -- 172.21.15.82:0/3438793929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 msgr2=0x5647a8ffc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864dbdd700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ffc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.786 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864dbdd700 1 -- 172.21.15.82:0/3438793929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 msgr2=0x5647a8ff9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864dbdd700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864dbdd700 1 -- 172.21.15.82:0/3438793929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864810c240 con 0x5647a8ff0790 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864e3de700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ffc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f864dbdd700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff95d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8638096880 tx=0x7f86380998a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d095172615f4896d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f8631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f8631ffb700 1 -- 172.21.15.82:0/3438793929 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863809a270 con 0x5647a8ff0790 2023-07-14T19:28:56.787 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f8631ffb700 1 -- 172.21.15.82:0/3438793929 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f8638099e10 con 0x5647a8ff0790 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647a8e9a4f0 con 0x5647a8ff0790 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f8631ffb700 1 -- 172.21.15.82:0/3438793929 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f863809de40 con 0x5647a8ff0790 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f865c7dab80 10 client.25262 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5647a8d57c60 con 0x5647a8ff0790 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f8631ffb700 1 -- 172.21.15.82:0/3438793929 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86380c8080 con 0x5647a8ff0790 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.778+0000 7f8631ffb700 1 -- 172.21.15.82:0/3438793929 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f86380a1dd0 con 0x5647a8ff0790 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f865c7dab80 10 client.25262 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:56.788 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f865c7dab80 10 client.25262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f865c7dab80 20 client.25262 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f865c7dab80 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a90077b0 0x5647a9010680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f865c7dab80 4 client.25262 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647a8fbeda0 con 0x5647a90077b0 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f864e3de700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a90077b0 0x5647a9010680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f8631ffb700 10 client.25262 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:56.789 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.779+0000 7f864e3de700 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a90077b0 0x5647a9010680 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=444d068b289e51d6 server_cookie=1f96f047de96aede in_seq=0 out_seq=0 2023-07-14T19:28:56.790 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f8631ffb700 1 -- 172.21.15.82:0/3438793929 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5647a8fbeda0 con 0x5647a90077b0 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f8631ffb700 10 client.25262 handle_command_reply: tid=0 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 client.25262 shutdown 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 msgr2=0x5647a8ff95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff95d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8638096880 tx=0x7f86380998a0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 shutdown_connections 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff4c30 0x5647a8ff9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.791 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a90077b0 0x5647a9010680 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8ffc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f8631ffb700 0 client.25262 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 --2- 172.21.15.82:0/3438793929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ff95d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 >> 172.21.15.82:0/3438793929 conn(0x5647a8fe5680 msgr2=0x5647a8feaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 shutdown_connections 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 1 -- 172.21.15.82:0/3438793929 wait complete. 2023-07-14T19:28:56.792 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.780+0000 7f865c7dab80 20 client.25262 trim_cache size 0 max 16384 2023-07-14T19:28:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 Processor -- start 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 -- start start 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 0x5647a8fe98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ffbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8ff4c30 0x5647a8ff8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8fe2a80 con 0x5647a8fefe80 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8ff7bb0 con 0x5647a8ff0790 2023-07-14T19:28:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f865c7dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a8ff3510 con 0x5647a8ff4c30 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f864e3de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 0x5647a8fe98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f864e3de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 0x5647a8fe98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42942/0 (socket says 172.21.15.82:42942) 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.810+0000 7f864e3de700 1 -- 172.21.15.82:0/2947873734 learned_addr learned my addr 172.21.15.82:0/2947873734 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864dbdd700 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ffbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864ebdf700 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8ff4c30 0x5647a8ff8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864e3de700 1 -- 172.21.15.82:0/2947873734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8ff4c30 msgr2=0x5647a8ff8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864e3de700 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8ff4c30 0x5647a8ff8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864e3de700 1 -- 172.21.15.82:0/2947873734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 msgr2=0x5647a8ffbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864e3de700 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ffbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864e3de700 1 -- 172.21.15.82:0/2947873734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a8d95250 con 0x5647a8fefe80 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f864e3de700 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 0x5647a8fe98b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f86440046e0 tx=0x7f864400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5603b1ee411fd945 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f863d7fa700 1 -- 172.21.15.82:0/2947873734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644022040 con 0x5647a8fefe80 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f863d7fa700 1 -- 172.21.15.82:0/2947873734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86440242c0 con 0x5647a8fefe80 2023-07-14T19:28:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.811+0000 7f863d7fa700 1 -- 172.21.15.82:0/2947873734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644024e60 con 0x5647a8fefe80 2023-07-14T19:28:56.817 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 -- 172.21.15.82:0/2947873734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 msgr2=0x5647a8fe98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 0x5647a8fe98b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f86440046e0 tx=0x7f864400af90 comp rx=0 tx=0).stop 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 -- 172.21.15.82:0/2947873734 shutdown_connections 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8ff4c30 0x5647a8ff8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8fefe80 0x5647a8fe98b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 --2- 172.21.15.82:0/2947873734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff0790 0x5647a8ffbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 -- 172.21.15.82:0/2947873734 >> 172.21.15.82:0/2947873734 conn(0x5647a8fe5680 msgr2=0x5647a8ff7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 -- 172.21.15.82:0/2947873734 shutdown_connections 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 -- 172.21.15.82:0/2947873734 wait complete. 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 -- start start 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.812+0000 7f865c7dab80 1 Processor -- start 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8fff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 0x5647a8fff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff4c30 0x5647a8fffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a9061bb0 con 0x5647a8ff0790 2023-07-14T19:28:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a9061cf0 con 0x5647a8ff4c30 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a9061e30 con 0x5647a8fefe80 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 10 client.0 _ll_register_callbacks cb 0x7f8638099be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 0x5647a8fff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 0x5647a8fff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42950/0 (socket says 172.21.15.82:42950) 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 -- 172.21.15.82:0/505229560 learned_addr learned my addr 172.21.15.82:0/505229560 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864ebdf700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff4c30 0x5647a8fffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.820 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864e3de700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8fff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 -- 172.21.15.82:0/505229560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 msgr2=0x5647a8fff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8fff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 -- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff4c30 msgr2=0x5647a8fffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff4c30 0x5647a8fffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 -- 172.21.15.82:0/505229560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647a8d6d2b0 con 0x5647a8ff0790 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f864dbdd700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 0x5647a8fff7c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8638007fd0 tx=0x7f86380a2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e2802a050ee344 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f8631ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:56.821 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f8631ffb700 1 -- 172.21.15.82:0/505229560 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863809a270 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f8631ffb700 1 -- 172.21.15.82:0/505229560 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f8638099e10 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.813+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86400092d0 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f8631ffb700 1 -- 172.21.15.82:0/505229560 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86380ca480 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 10 client.15537 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5647a8e3ba80 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f8631ffb700 1 -- 172.21.15.82:0/505229560 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86400092d0 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f8631ffb700 1 -- 172.21.15.82:0/505229560 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f86380b2390 con 0x5647a8ff0790 2023-07-14T19:28:56.822 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 10 client.15537 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 20 client.15537 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a9063200 0x5647a9065610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 4 client.15537 mds_command: new command op to 24524 tid=0[{"prefix": "scrub status"}] 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647a8f39eb0 con 0x5647a9063200 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.814+0000 7f864e3de700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a9063200 0x5647a9065610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.823 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f8631ffb700 10 client.15537 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f864e3de700 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a9063200 0x5647a9065610 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4e23afa6d96f8d3 server_cookie=7c146c1b513b19f8 in_seq=0 out_seq=0 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f8631ffb700 1 -- 172.21.15.82:0/505229560 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5647a8f39eb0 con 0x5647a9063200 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f8631ffb700 10 client.15537 handle_command_reply: tid=0 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 client.15537 shutdown 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 msgr2=0x5647a8fff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 0x5647a8fff7c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8638007fd0 tx=0x7f86380a2000 comp rx=0 tx=0).stop 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 shutdown_connections 2023-07-14T19:28:56.824 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647a8ff4c30 0x5647a8fffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5647a9063200 0x5647a9065610 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5647a8fefe80 0x5647a8fff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 --2- 172.21.15.82:0/505229560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647a8ff0790 0x5647a8fff7c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 >> 172.21.15.82:0/505229560 conn(0x5647a8fe5680 msgr2=0x5647a8fea0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f8631ffb700 0 client.15537 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 shutdown_connections 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 1 -- 172.21.15.82:0/505229560 wait complete. 2023-07-14T19:28:56.825 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.815+0000 7f865c7dab80 20 client.15537 trim_cache size 0 max 16384 2023-07-14T19:28:56.826 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2023-07-14T19:28:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2023-07-14T19:28:56.826 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f86340a1350 msgr2=0x7f86340a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f86340a1350 0x7f86340a3800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f864400abd0 tx=0x7f864400a380 comp rx=0 tx=0).stop 2023-07-14T19:28:56.827 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 msgr2=0x7f864819af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864819af00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f863800c9b0 tx=0x7f863800ccc0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 shutdown_connections 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f86340a1350 0x7f86340a3800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8648104b10 0x7f8648194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86480feb00 0x7f864819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 --2- 172.21.15.82:0/4125936659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86480ff4b0 0x7f864819af00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 >> 172.21.15.82:0/4125936659 conn(0x7f86480fa5e0 msgr2=0x7f8648102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 shutdown_connections 2023-07-14T19:28:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.816+0000 7f8650642700 1 -- 172.21.15.82:0/4125936659 wait complete. 2023-07-14T19:28:56.829 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T19:28:56.830 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:28:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 -- 172.21.15.82:0/763819797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c103ac0 msgr2=0x7f263c103e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 --2- 172.21.15.82:0/763819797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c103ac0 0x7f263c103e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f263800b0d0 tx=0x7f263800b3e0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 -- 172.21.15.82:0/763819797 shutdown_connections 2023-07-14T19:28:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 --2- 172.21.15.82:0/763819797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 0x7f263c10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 --2- 172.21.15.82:0/763819797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c103ac0 0x7f263c103e90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 --2- 172.21.15.82:0/763819797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263c0ff270 0x7f263c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 -- 172.21.15.82:0/763819797 >> 172.21.15.82:0/763819797 conn(0x7f263c074f30 msgr2=0x7f263c075330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.959+0000 7f2644250700 1 -- 172.21.15.82:0/763819797 shutdown_connections 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 -- 172.21.15.82:0/763819797 wait complete. 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 Processor -- start 2023-07-14T19:28:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 -- start start 2023-07-14T19:28:56.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 0x7f263c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 0x7f263c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263c103ac0 0x7f263c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19aea0 con 0x7f263c0ff270 2023-07-14T19:28:56.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19b010 con 0x7f263c103ac0 2023-07-14T19:28:56.964 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2644250700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c19b1d0 con 0x7f263c0ffc20 2023-07-14T19:28:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2641fec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 0x7f263c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2641fec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 0x7f263c195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42962/0 (socket says 172.21.15.82:42962) 2023-07-14T19:28:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.960+0000 7f2641fec700 1 -- 172.21.15.82:0/119238069 learned_addr learned my addr 172.21.15.82:0/119238069 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f26417eb700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 0x7f263c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2641fec700 1 -- 172.21.15.82:0/119238069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 msgr2=0x7f263c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2641fec700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 0x7f263c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2641fec700 1 -- 172.21.15.82:0/119238069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263c103ac0 msgr2=0x7f263c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2641fec700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263c103ac0 0x7f263c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2641fec700 1 -- 172.21.15.82:0/119238069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638009d00 con 0x7f263c0ff270 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f26417eb700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 0x7f263c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2641fec700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 0x7f263c195ec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2638000f80 tx=0x7f2638009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263800e040 con 0x7f263c0ff270 2023-07-14T19:28:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2638012700 con 0x7f263c0ff270 2023-07-14T19:28:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c19b450 con 0x7f263c0ff270 2023-07-14T19:28:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26380053c0 con 0x7f263c0ff270 2023-07-14T19:28:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.961+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c1a2cf0 con 0x7f263c0ff270 2023-07-14T19:28:56.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.962+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c066ba0 con 0x7f263c0ff270 2023-07-14T19:28:56.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.963+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f2638012870 con 0x7f263c0ff270 2023-07-14T19:28:56.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:56 smithi082 ceph-mon[127461]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-14T19:28:56.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.964+0000 7f2632ffd700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f26280a1390 0x7f26280a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:56.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.964+0000 7f26417eb700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f26280a1390 0x7f26280a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:56.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.964+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f26380ce710 con 0x7f263c0ff270 2023-07-14T19:28:56.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.964+0000 7f26417eb700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f26280a1390 0x7f26280a3840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f263c1974d0 tx=0x7f262c00b5f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:56.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:56.968+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2638097ad0 con 0x7f263c0ff270 2023-07-14T19:28:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:56 smithi161 ceph-mon[129184]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-14T19:28:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:56 smithi133 ceph-mon[128894]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-07-14T19:28:57.208 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.208+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f263c197210 con 0x7f263c0ff270 2023-07-14T19:28:57.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.208+0000 7f2632ffd700 1 -- 172.21.15.82:0/119238069 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f263809b0f0 con 0x7f263c0ff270 2023-07-14T19:28:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:28:57.209 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:28:56.700827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/42384414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":42384414},{"type":"v1","addr":"172.21.15.161:6837","nonce":42384414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:28:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f26280a1390 msgr2=0x7f26280a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f26280a1390 0x7f26280a3840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f263c1974d0 tx=0x7f262c00b5f0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 msgr2=0x7f263c195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 0x7f263c195ec0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2638000f80 tx=0x7f2638009410 comp rx=0 tx=0).stop 2023-07-14T19:28:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 shutdown_connections 2023-07-14T19:28:57.212 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f26280a1390 0x7f26280a3840 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f263c103ac0 0x7f263c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f263c0ff270 0x7f263c195ec0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 --2- 172.21.15.82:0/119238069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f263c0ffc20 0x7f263c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 >> 172.21.15.82:0/119238069 conn(0x7f263c074f30 msgr2=0x7f263c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 shutdown_connections 2023-07-14T19:28:57.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.211+0000 7f2644250700 1 -- 172.21.15.82:0/119238069 wait complete. 2023-07-14T19:28:57.214 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2023-07-14T19:28:57.228 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T19:28:57.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/1161725287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec80fe9d0 msgr2=0x7f9ec80feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/1161725287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec80fe9d0 0x7f9ec80feda0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009990 tx=0x7f9ec4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/1161725287 shutdown_connections 2023-07-14T19:28:57.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/1161725287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec810c580 0x7f9ec810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/1161725287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec80fe9d0 0x7f9ec80feda0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/1161725287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec80ff2e0 0x7f9ec810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/1161725287 >> 172.21.15.82:0/1161725287 conn(0x7f9ec80fa5e0 msgr2=0x7f9ec80fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/1161725287 shutdown_connections 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/1161725287 wait complete. 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 Processor -- start 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.357+0000 7f9ecf4b1700 1 -- start start 2023-07-14T19:28:57.360 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec80fe9d0 0x7f9ec8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec80ff2e0 0x7f9ec8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 0x7f9ec819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec819cf30 con 0x7f9ec810c580 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec819d0a0 con 0x7f9ec80ff2e0 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec819d260 con 0x7f9ec80fe9d0 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 0x7f9ec819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 0x7f9ec819c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42980/0 (socket says 172.21.15.82:42980) 2023-07-14T19:28:57.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 -- 172.21.15.82:0/2852090431 learned_addr learned my addr 172.21.15.82:0/2852090431 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec80fe9d0 0x7f9ec8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec80ff2e0 0x7f9ec8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 -- 172.21.15.82:0/2852090431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec80fe9d0 msgr2=0x7f9ec8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec80fe9d0 0x7f9ec8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.362 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 -- 172.21.15.82:0/2852090431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec80ff2e0 msgr2=0x7f9ec8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec80ff2e0 0x7f9ec8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 -- 172.21.15.82:0/2852090431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec40095f0 con 0x7f9ec810c580 2023-07-14T19:28:57.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 0x7f9ec819c8f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0007ce0 tx=0x7f9ec000d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ebe7fc700 1 -- 172.21.15.82:0/2852090431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0004db0 con 0x7f9ec810c580 2023-07-14T19:28:57.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec819d540 con 0x7f9ec810c580 2023-07-14T19:28:57.363 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ebe7fc700 1 -- 172.21.15.82:0/2852090431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec000f040 con 0x7f9ec810c580 2023-07-14T19:28:57.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.358+0000 7f9ebe7fc700 1 -- 172.21.15.82:0/2852090431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00059f0 con 0x7f9ec810c580 2023-07-14T19:28:57.364 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.359+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec81a4d80 con 0x7f9ec810c580 2023-07-14T19:28:57.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.361+0000 7f9ebe7fc700 1 -- 172.21.15.82:0/2852090431 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f9ec00050b0 con 0x7f9ec810c580 2023-07-14T19:28:57.368 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.361+0000 7f9ebe7fc700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9eb40a1350 0x7f9eb40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.361+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9eb40a1350 0x7f9eb40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.362+0000 7f9ebe7fc700 1 -- 172.21.15.82:0/2852090431 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9ec00cc680 con 0x7f9ec810c580 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.362+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9eb40a1350 0x7f9eb40a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400b5c0 tx=0x7f9ec4005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 Processor -- start 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 -- start start 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b05f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b0636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b064080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1af63cb0 con 0x556a1b05f790 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1afb6a40 con 0x556a1b063c30 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9edb649b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1afafed0 con 0x556a1b05ee80 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9ecca4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b0636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9ecca4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b0636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42986/0 (socket says 172.21.15.82:42986) 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9ecca4c700 1 -- 172.21.15.82:0/2038981111 learned_addr learned my addr 172.21.15.82:0/2038981111 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.370 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b05f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.363+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b064080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9ecca4c700 1 -- 172.21.15.82:0/2038981111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 msgr2=0x556a1b05f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b05f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9ecca4c700 1 -- 172.21.15.82:0/2038981111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 msgr2=0x556a1b064080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.371 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b064080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9ecca4c700 1 -- 172.21.15.82:0/2038981111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a1adddde0 con 0x556a1b05f790 2023-07-14T19:28:57.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b0636f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009bf0 tx=0x7f9eb8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0685fc70755c13 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9eb37fe700 1 -- 172.21.15.82:0/2038981111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb800eaa0 con 0x556a1b05f790 2023-07-14T19:28:57.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9eb37fe700 1 -- 172.21.15.82:0/2038981111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eb8004cf0 con 0x556a1b05f790 2023-07-14T19:28:57.372 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.364+0000 7f9eb37fe700 1 -- 172.21.15.82:0/2038981111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8005640 con 0x556a1b05f790 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 -- 172.21.15.82:0/2038981111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 msgr2=0x556a1b0636f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b0636f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009bf0 tx=0x7f9eb8009f00 comp rx=0 tx=0).stop 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 -- 172.21.15.82:0/2038981111 shutdown_connections 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b064080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b05f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 --2- 172.21.15.82:0/2038981111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b0636f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.373 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 -- 172.21.15.82:0/2038981111 >> 172.21.15.82:0/2038981111 conn(0x556a1b054680 msgr2=0x556a1b059ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 -- 172.21.15.82:0/2038981111 shutdown_connections 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 -- 172.21.15.82:0/2038981111 wait complete. 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 -- start start 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.365+0000 7f9edb649b80 1 Processor -- start 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b068720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b068c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b0691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.374 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1af4fac0 con 0x556a1b05f790 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1af4f7c0 con 0x556a1b063c30 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1ae33920 con 0x556a1b05ee80 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9edb649b80 10 client.0 _ll_register_callbacks cb 0x556a1af4d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b068c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b068c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42992/0 (socket says 172.21.15.82:42992) 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 -- 172.21.15.82:0/80540416 learned_addr learned my addr 172.21.15.82:0/80540416 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.375 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b068720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b0691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 -- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 msgr2=0x556a1b068720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b068720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 -- 172.21.15.82:0/80540416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 msgr2=0x556a1b0691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b0691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.366+0000 7f9ecca4c700 1 -- 172.21.15.82:0/80540416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb8009820 con 0x556a1b05f790 2023-07-14T19:28:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b068c60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8000c00 tx=0x7f9eb8005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11a85dc042dac92 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9eb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/80540416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8021070 con 0x556a1b05f790 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/80540416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9eb800e8a0 con 0x556a1b05f790 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a1af094f0 con 0x556a1b05f790 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/80540416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eb8004fe0 con 0x556a1b05f790 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 10 client.15555 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a1adc6c60 con 0x556a1b05f790 2023-07-14T19:28:57.377 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/80540416 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556a1af094f0 con 0x556a1b05f790 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/80540416 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9eb800c070 con 0x556a1b05f790 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 10 client.15555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 20 client.15555 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b078bf0 0x556a1b07b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 4 client.15555 mds_command: new command op to 24481 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.367+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a1b02dda0 con 0x556a1b078bf0 2023-07-14T19:28:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.368+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b078bf0 0x556a1b07b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.368+0000 7f9eb0ff9700 10 client.15555 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.368+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b078bf0 0x556a1b07b000 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=706573e467a32916 server_cookie=74ee9fe068cbd895 in_seq=0 out_seq=0 2023-07-14T19:28:57.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/80540416 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556a1b02dda0 con 0x556a1b078bf0 2023-07-14T19:28:57.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9eb0ff9700 10 client.15555 handle_command_reply: tid=0 2023-07-14T19:28:57.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 client.15555 shutdown 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 msgr2=0x556a1b068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b068c60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8000c00 tx=0x7f9eb8005470 comp rx=0 tx=0).stop 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 shutdown_connections 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b063c30 0x556a1b0691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b078bf0 0x556a1b07b000 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b068720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9eb0ff9700 0 client.15555 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:57.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 --2- 172.21.15.82:0/80540416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b05f790 0x556a1b068c60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 >> 172.21.15.82:0/80540416 conn(0x556a1b054680 msgr2=0x556a1b059a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 shutdown_connections 2023-07-14T19:28:57.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 1 -- 172.21.15.82:0/80540416 wait complete. 2023-07-14T19:28:57.382 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.369+0000 7f9edb649b80 20 client.15555 trim_cache size 0 max 16384 2023-07-14T19:28:57.399 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 Processor -- start 2023-07-14T19:28:57.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 -- start start 2023-07-14T19:28:57.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05ee80 0x556a1b061520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05f790 0x556a1b06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b061e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1b06b8a0 con 0x556a1b063c30 2023-07-14T19:28:57.400 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1b05a420 con 0x556a1b05ee80 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.395+0000 7f9edb649b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1b058cd0 con 0x556a1b05f790 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b061e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b061e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43002/0 (socket says 172.21.15.82:43002) 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 -- 172.21.15.82:0/1009728273 learned_addr learned my addr 172.21.15.82:0/1009728273 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05f790 0x556a1b06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05ee80 0x556a1b061520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.401 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 -- 172.21.15.82:0/1009728273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05f790 msgr2=0x556a1b06fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05f790 0x556a1b06fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 -- 172.21.15.82:0/1009728273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05ee80 msgr2=0x556a1b061520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05ee80 0x556a1b061520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 -- 172.21.15.82:0/1009728273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a1b068290 con 0x556a1b063c30 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05ee80 0x556a1b061520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b061e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00c9e30 tx=0x7f9ec00b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46af68f33169fe58 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9eb3fff700 1 -- 172.21.15.82:0/1009728273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec009d6a0 con 0x556a1b063c30 2023-07-14T19:28:57.402 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9eb3fff700 1 -- 172.21.15.82:0/1009728273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec00a1ac0 con 0x556a1b063c30 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.396+0000 7f9eb3fff700 1 -- 172.21.15.82:0/1009728273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0084310 con 0x556a1b063c30 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 -- 172.21.15.82:0/1009728273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 msgr2=0x556a1b061e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b061e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00c9e30 tx=0x7f9ec00b9040 comp rx=0 tx=0).stop 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 -- 172.21.15.82:0/1009728273 shutdown_connections 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b061e00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05ee80 0x556a1b061520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 --2- 172.21.15.82:0/1009728273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05f790 0x556a1b06fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.403 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 -- 172.21.15.82:0/1009728273 >> 172.21.15.82:0/1009728273 conn(0x556a1b054680 msgr2=0x556a1b074390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 -- 172.21.15.82:0/1009728273 shutdown_connections 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 -- 172.21.15.82:0/1009728273 wait complete. 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 -- start start 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.397+0000 7f9edb649b80 1 Processor -- start 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05f790 0x556a1b06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b06de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.404 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1b06e5d0 con 0x556a1b063c30 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1b06e740 con 0x556a1b05f790 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a1b06e900 con 0x556a1b05ee80 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 10 client.0 _ll_register_callbacks cb 0x7f9ec00c9c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b06de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b06de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43008/0 (socket says 172.21.15.82:43008) 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 -- 172.21.15.82:0/3869622044 learned_addr learned my addr 172.21.15.82:0/3869622044 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.405 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05f790 0x556a1b06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 -- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 msgr2=0x556a1b06f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 -- 172.21.15.82:0/3869622044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05f790 msgr2=0x556a1b06fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05f790 0x556a1b06fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 -- 172.21.15.82:0/3869622044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec0082e70 con 0x556a1b063c30 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecca4c700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05f790 0x556a1b06fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:57.406 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9ecda4e700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b06de90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00ca460 tx=0x7f9ec0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a2c962d61e592a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9eb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:57.407 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/3869622044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00c3ea0 con 0x556a1b063c30 2023-07-14T19:28:57.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9eb8004cf0 con 0x556a1b063c30 2023-07-14T19:28:57.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.398+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/3869622044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9ec0097ad0 con 0x556a1b063c30 2023-07-14T19:28:57.408 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/3869622044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ec00998d0 con 0x556a1b063c30 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 10 client.15561 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a1ae04250 con 0x556a1b063c30 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/3869622044 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec0063080 con 0x556a1b063c30 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/3869622044 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f9ec00a16c0 con 0x556a1b063c30 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 10 client.15561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 20 client.15561 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:57.409 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b0cd7a0 0x556a1b0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 4 client.15561 mds_command: new command op to 24481 tid=0[{"prefix": "damage ls"}] 2023-07-14T19:28:57.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a1afa8eb0 con 0x556a1b0cd7a0 2023-07-14T19:28:57.410 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.399+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b0cd7a0 0x556a1b0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.411 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9eb0ff9700 10 client.15561 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9ecd24d700 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b0cd7a0 0x556a1b0cfb80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=feecd287e0344af0 server_cookie=3551ce63c6438ee6 in_seq=0 out_seq=0 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9eb0ff9700 1 -- 172.21.15.82:0/3869622044 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a1afa8eb0 con 0x556a1b0cd7a0 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9eb0ff9700 10 client.15561 handle_command_reply: tid=0 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 client.15561 shutdown 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 msgr2=0x556a1b06de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b06de90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00ca460 tx=0x7f9ec0009540 comp rx=0 tx=0).stop 2023-07-14T19:28:57.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 shutdown_connections 2023-07-14T19:28:57.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a1b063c30 0x556a1b06de90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x556a1b0cd7a0 0x556a1b0cfb80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556a1b05ee80 0x556a1b06f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 --2- 172.21.15.82:0/3869622044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556a1b05f790 0x556a1b06fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 >> 172.21.15.82:0/3869622044 conn(0x556a1b054680 msgr2=0x556a1b061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.413 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9eb0ff9700 0 client.15561 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:28:57.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 shutdown_connections 2023-07-14T19:28:57.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 1 -- 172.21.15.82:0/3869622044 wait complete. 2023-07-14T19:28:57.414 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.400+0000 7f9edb649b80 20 client.15561 trim_cache size 0 max 16384 2023-07-14T19:28:57.415 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-07-14T19:28:57.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9eb40a1350 msgr2=0x7f9eb40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9eb40a1350 0x7f9eb40a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400b5c0 tx=0x7f9ec4005fd0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.416 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 msgr2=0x7f9ec819c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 0x7f9ec819c8f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0007ce0 tx=0x7f9ec000d0d0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 shutdown_connections 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9eb40a1350 0x7f9eb40a3800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ec810c580 0x7f9ec819c8f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec80fe9d0 0x7f9ec8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 --2- 172.21.15.82:0/2852090431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec80ff2e0 0x7f9ec8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.417 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 >> 172.21.15.82:0/2852090431 conn(0x7f9ec80fa5e0 msgr2=0x7f9ec810da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 shutdown_connections 2023-07-14T19:28:57.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.401+0000 7f9ecf4b1700 1 -- 172.21.15.82:0/2852090431 wait complete. 2023-07-14T19:28:57.418 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T19:28:57.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 -- 172.21.15.82:0/2949225128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 msgr2=0x7f52ec10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 --2- 172.21.15.82:0/2949225128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec10cad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b0d0 tx=0x7f52e400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 -- 172.21.15.82:0/2949225128 shutdown_connections 2023-07-14T19:28:57.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 --2- 172.21.15.82:0/2949225128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec10cad0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 --2- 172.21.15.82:0/2949225128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 0x7f52ec100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 --2- 172.21.15.82:0/2949225128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100860 0x7f52ec104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 -- 172.21.15.82:0/2949225128 >> 172.21.15.82:0/2949225128 conn(0x7f52ec0fb810 msgr2=0x7f52ec0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 -- 172.21.15.82:0/2949225128 shutdown_connections 2023-07-14T19:28:57.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.546+0000 7f52f205b700 1 -- 172.21.15.82:0/2949225128 wait complete. 2023-07-14T19:28:57.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.547+0000 7f52f205b700 1 Processor -- start 2023-07-14T19:28:57.550 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.547+0000 7f52f205b700 1 -- start start 2023-07-14T19:28:57.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.547+0000 7f52f205b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 0x7f52ec195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.547+0000 7f52f205b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100860 0x7f52ec1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.547+0000 7f52f205b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52f205b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19ae90 con 0x7f52ec104f00 2023-07-14T19:28:57.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52f205b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19b000 con 0x7f52ec0ffec0 2023-07-14T19:28:57.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52f205b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec19b1c0 con 0x7f52ec100860 2023-07-14T19:28:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43022/0 (socket says 172.21.15.82:43022) 2023-07-14T19:28:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 -- 172.21.15.82:0/3343454813 learned_addr learned my addr 172.21.15.82:0/3343454813 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52eaffd700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100860 0x7f52ec1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 0x7f52ec195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 -- 172.21.15.82:0/3343454813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100860 msgr2=0x7f52ec1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100860 0x7f52ec1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 -- 172.21.15.82:0/3343454813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 msgr2=0x7f52ec195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 0x7f52ec195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 -- 172.21.15.82:0/3343454813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009d00 con 0x7f52ec104f00 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 0x7f52ec195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:57.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52ebfff700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec19a850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f52e4007870 tx=0x7f52e40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52e8ff9700 1 -- 172.21.15.82:0/3343454813 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e400e040 con 0x7f52ec104f00 2023-07-14T19:28:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52e8ff9700 1 -- 172.21.15.82:0/3343454813 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e4005070 con 0x7f52ec104f00 2023-07-14T19:28:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec19b440 con 0x7f52ec104f00 2023-07-14T19:28:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52e8ff9700 1 -- 172.21.15.82:0/3343454813 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e401b6a0 con 0x7f52ec104f00 2023-07-14T19:28:57.554 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.548+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1a2ce0 con 0x7f52ec104f00 2023-07-14T19:28:57.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.550+0000 7f52e8ff9700 1 -- 172.21.15.82:0/3343454813 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f52e40051e0 con 0x7f52ec104f00 2023-07-14T19:28:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.551+0000 7f52e8ff9700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f52d40a1350 0x7f52d40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.551+0000 7f52e8ff9700 1 -- 172.21.15.82:0/3343454813 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f52e40ce000 con 0x7f52ec104f00 2023-07-14T19:28:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.551+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f52d40a1350 0x7f52d40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.551+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f52d40a1350 0x7f52d40a3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00bc10 tx=0x7f52dc00b5f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 Processor -- start 2023-07-14T19:28:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 -- start start 2023-07-14T19:28:57.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f0a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f0e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7e0ecb0 con 0x5586a7f0ec30 2023-07-14T19:28:57.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7e61a40 con 0x5586a7f09e80 2023-07-14T19:28:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7e5aed0 con 0x5586a7f0a790 2023-07-14T19:28:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f0a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f0a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:39374/0 (socket says 172.21.15.82:39374) 2023-07-14T19:28:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f0f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43030/0 (socket says 172.21.15.82:43030) 2023-07-14T19:28:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52eb7fe700 1 -- 172.21.15.82:0/1076566916 learned_addr learned my addr 172.21.15.82:0/1076566916 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.553+0000 7f52eaffd700 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f0e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52ebfff700 1 -- 172.21.15.82:0/1076566916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 msgr2=0x5586a7f0e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52ebfff700 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f0e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52ebfff700 1 -- 172.21.15.82:0/1076566916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 msgr2=0x5586a7f0a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52ebfff700 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f0a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52ebfff700 1 -- 172.21.15.82:0/1076566916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586a7c88de0 con 0x5586a7f0ec30 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f0a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.554+0000 7f52ebfff700 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f0f080 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f52e40cb720 tx=0x7f52e409af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329e3dff1b781fee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.555+0000 7f52d1ffb700 1 -- 172.21.15.82:0/1076566916 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e40054e0 con 0x5586a7f0ec30 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.555+0000 7f52d1ffb700 1 -- 172.21.15.82:0/1076566916 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e409b080 con 0x5586a7f0ec30 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.555+0000 7f52d1ffb700 1 -- 172.21.15.82:0/1076566916 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4003c60 con 0x5586a7f0ec30 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1076566916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 msgr2=0x5586a7f0f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f0f080 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f52e40cb720 tx=0x7f52e409af20 comp rx=0 tx=0).stop 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1076566916 shutdown_connections 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f0f080 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f0a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1076566916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f0e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1076566916 >> 172.21.15.82:0/1076566916 conn(0x5586a7eff680 msgr2=0x5586a7f04ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1076566916 shutdown_connections 2023-07-14T19:28:57.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1076566916 wait complete. 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.556+0000 7f52fe1f3b80 1 -- start start 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 Processor -- start 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 0x5586a7f16350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0ec30 0x5586a7f13c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7dfaac0 con 0x5586a7f09e80 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7dfa7c0 con 0x5586a7f0ec30 2023-07-14T19:28:57.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7cde920 con 0x5586a7f0a790 2023-07-14T19:28:57.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52fe1f3b80 10 client.0 _ll_register_callbacks cb 0x5586a7df8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:57.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 0x5586a7f16350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 0x5586a7f16350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43032/0 (socket says 172.21.15.82:43032) 2023-07-14T19:28:57.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 -- 172.21.15.82:0/1349146664 learned_addr learned my addr 172.21.15.82:0/1349146664 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.572 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52ebfff700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0ec30 0x5586a7f13c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eaffd700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 -- 172.21.15.82:0/1349146664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 msgr2=0x5586a7f135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 -- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0ec30 msgr2=0x5586a7f13c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.573 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0ec30 0x5586a7f13c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 -- 172.21.15.82:0/1349146664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc00f9f0 con 0x5586a7f09e80 2023-07-14T19:28:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52ebfff700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0ec30 0x5586a7f13c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 0x5586a7f16350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0116f0 tx=0x7f52dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faad1fd90d81cbf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.574 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.557+0000 7f52caffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52caffd700 1 -- 172.21.15.82:0/1349146664 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc01f840 con 0x5586a7f09e80 2023-07-14T19:28:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52caffd700 1 -- 172.21.15.82:0/1349146664 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f52dc01f9a0 con 0x5586a7f09e80 2023-07-14T19:28:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586a7db44f0 con 0x5586a7f09e80 2023-07-14T19:28:57.575 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52caffd700 1 -- 172.21.15.82:0/1349146664 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc01fb60 con 0x5586a7f09e80 2023-07-14T19:28:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 10 client.15573 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f52ec1a2ce0 con 0x5586a7f09e80 2023-07-14T19:28:57.576 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52caffd700 1 -- 172.21.15.82:0/1349146664 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5586a7db44f0 con 0x5586a7f09e80 2023-07-14T19:28:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52caffd700 1 -- 172.21.15.82:0/1349146664 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f52dc01e430 con 0x5586a7f09e80 2023-07-14T19:28:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 10 client.15573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-14T19:28:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 20 client.15573 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f23a10 0x5586a7f25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 4 client.15573 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.558+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586a7ed8da0 con 0x5586a7f23a10 2023-07-14T19:28:57.578 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.559+0000 7f52eaffd700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f23a10 0x5586a7f25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.559+0000 7f52caffd700 10 client.15573 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.559+0000 7f52eaffd700 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f23a10 0x5586a7f25e20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b06818b242bd3d3d server_cookie=db98ba07a15b500e in_seq=0 out_seq=0 2023-07-14T19:28:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52caffd700 1 -- 172.21.15.82:0/1349146664 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5586a7ed8da0 con 0x5586a7f23a10 2023-07-14T19:28:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52caffd700 10 client.15573 handle_command_reply: tid=0 2023-07-14T19:28:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 client.15573 shutdown 2023-07-14T19:28:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 msgr2=0x5586a7f16350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 0x5586a7f16350 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0116f0 tx=0x7f52dc004750 comp rx=0 tx=0).stop 2023-07-14T19:28:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 shutdown_connections 2023-07-14T19:28:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f23a10 0x5586a7f25e20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0ec30 0x5586a7f13c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f09e80 0x5586a7f16350 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52caffd700 0 client.15573 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/1349146664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0a790 0x5586a7f135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 >> 172.21.15.82:0/1349146664 conn(0x5586a7eff680 msgr2=0x5586a7f04a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 shutdown_connections 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/1349146664 wait complete. 2023-07-14T19:28:57.581 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.560+0000 7f52fe1f3b80 20 client.15573 trim_cache size 0 max 16384 2023-07-14T19:28:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 Processor -- start 2023-07-14T19:28:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 -- start start 2023-07-14T19:28:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 0x5586a7f10010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0ec30 0x5586a7f15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7f06550 con 0x5586a7f0a790 2023-07-14T19:28:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7f21110 con 0x5586a7f09e80 2023-07-14T19:28:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7f0af40 con 0x5586a7f0ec30 2023-07-14T19:28:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52eaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 0x5586a7f10010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52eaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 0x5586a7f10010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43042/0 (socket says 172.21.15.82:43042) 2023-07-14T19:28:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52eaffd700 1 -- 172.21.15.82:0/2114130904 learned_addr learned my addr 172.21.15.82:0/2114130904 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.573+0000 7f52ebfff700 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0ec30 0x5586a7f15ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eaffd700 1 -- 172.21.15.82:0/2114130904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0ec30 msgr2=0x5586a7f15ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eaffd700 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0ec30 0x5586a7f15ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eaffd700 1 -- 172.21.15.82:0/2114130904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 msgr2=0x5586a7f1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eaffd700 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eaffd700 1 -- 172.21.15.82:0/2114130904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586a7c71c60 con 0x5586a7f0a790 2023-07-14T19:28:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52eaffd700 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 0x5586a7f10010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52ec069c30 tx=0x7f52e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6745ef5f23b680a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52d27fc700 1 -- 172.21.15.82:0/2114130904 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e001f330 con 0x5586a7f0a790 2023-07-14T19:28:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52d27fc700 1 -- 172.21.15.82:0/2114130904 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e0018040 con 0x5586a7f0a790 2023-07-14T19:28:57.592 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.574+0000 7f52d27fc700 1 -- 172.21.15.82:0/2114130904 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0010430 con 0x5586a7f0a790 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2114130904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 msgr2=0x5586a7f10010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 0x5586a7f10010 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52ec069c30 tx=0x7f52e0004750 comp rx=0 tx=0).stop 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2114130904 shutdown_connections 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f0ec30 0x5586a7f15ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f09e80 0x5586a7f1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2114130904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0a790 0x5586a7f10010 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2114130904 >> 172.21.15.82:0/2114130904 conn(0x5586a7eff680 msgr2=0x5586a7f1f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2114130904 shutdown_connections 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2114130904 wait complete. 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 -- start start 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.575+0000 7f52fe1f3b80 1 Processor -- start 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f09e80 0x5586a7f13020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0a790 0x5586a7f13560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.594 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7f124b0 con 0x5586a7f0ec30 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7f12620 con 0x5586a7f0a790 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586a7f127e0 con 0x5586a7f09e80 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 10 client.0 _ll_register_callbacks cb 0x7f52dc00fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f11da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43050/0 (socket says 172.21.15.82:43050) 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 -- 172.21.15.82:0/2987863419 learned_addr learned my addr 172.21.15.82:0/2987863419 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.595 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f09e80 0x5586a7f13020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52eaffd700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0a790 0x5586a7f13560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 -- 172.21.15.82:0/2987863419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f09e80 msgr2=0x5586a7f13020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f09e80 0x5586a7f13020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 -- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0a790 msgr2=0x5586a7f13560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0a790 0x5586a7f13560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.596 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 -- 172.21.15.82:0/2987863419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e0009e20 con 0x5586a7f0ec30 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52eaffd700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0a790 0x5586a7f13560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52ebfff700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f11da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52e40cb800 tx=0x7f52e40cb6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892eb6be0f06652e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52caffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52caffd700 1 -- 172.21.15.82:0/2987863419 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e40cbbe0 con 0x5586a7f0ec30 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52caffd700 1 -- 172.21.15.82:0/2987863419 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f52e409f0b0 con 0x5586a7f0ec30 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52e409b080 con 0x5586a7f0ec30 2023-07-14T19:28:57.597 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.576+0000 7f52caffd700 1 -- 172.21.15.82:0/2987863419 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e40a3440 con 0x5586a7f0ec30 2023-07-14T19:28:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 10 client.15579 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5586a7caf250 con 0x5586a7f0ec30 2023-07-14T19:28:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52caffd700 1 -- 172.21.15.82:0/2987863419 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e409b080 con 0x5586a7f0ec30 2023-07-14T19:28:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52caffd700 1 -- 172.21.15.82:0/2987863419 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f52e409b2e0 con 0x5586a7f0ec30 2023-07-14T19:28:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:57.598 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 10 client.15579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 20 client.15579 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f78350 0x5586a7f7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 4 client.15579 mds_command: new command op to 24491 tid=0[{"prefix": "damage ls"}] 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5586a7e53eb0 con 0x5586a7f78350 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.577+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f78350 0x5586a7f7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52caffd700 10 client.15579 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:57.599 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52eb7fe700 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f78350 0x5586a7f7a760 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b59a6423ffe225e0 server_cookie=6af197308a37cc0 in_seq=0 out_seq=0 2023-07-14T19:28:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52caffd700 1 -- 172.21.15.82:0/2987863419 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5586a7e53eb0 con 0x5586a7f78350 2023-07-14T19:28:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52caffd700 10 client.15579 handle_command_reply: tid=0 2023-07-14T19:28:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 client.15579 shutdown 2023-07-14T19:28:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 msgr2=0x5586a7f11da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f11da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52e40cb800 tx=0x7f52e40cb6a0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.600 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 shutdown_connections 2023-07-14T19:28:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x5586a7f78350 0x5586a7f7a760 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5586a7f0ec30 0x5586a7f11da0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52caffd700 0 client.15579 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:28:57.601 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586a7f09e80 0x5586a7f13020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 --2- 172.21.15.82:0/2987863419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586a7f0a790 0x5586a7f13560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 >> 172.21.15.82:0/2987863419 conn(0x5586a7eff680 msgr2=0x5586a7f108c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 shutdown_connections 2023-07-14T19:28:57.602 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 1 -- 172.21.15.82:0/2987863419 wait complete. 2023-07-14T19:28:57.603 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.578+0000 7f52fe1f3b80 20 client.15579 trim_cache size 0 max 16384 2023-07-14T19:28:57.603 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-07-14T19:28:57.605 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f52d40a1350 msgr2=0x7f52d40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f52d40a1350 0x7f52d40a3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00bc10 tx=0x7f52dc00b5f0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 msgr2=0x7f52ec19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.606 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec19a850 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f52e4007870 tx=0x7f52e40078a0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 shutdown_connections 2023-07-14T19:28:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f52d40a1350 0x7f52d40a3800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52ec104f00 0x7f52ec19a850 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52ec0ffec0 0x7f52ec195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 --2- 172.21.15.82:0/3343454813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52ec100860 0x7f52ec1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 >> 172.21.15.82:0/3343454813 conn(0x7f52ec0fb810 msgr2=0x7f52ec102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 shutdown_connections 2023-07-14T19:28:57.608 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.579+0000 7f52f205b700 1 -- 172.21.15.82:0/3343454813 wait complete. 2023-07-14T19:28:57.608 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T19:28:57.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.735+0000 7fa87d9f4700 1 -- 172.21.15.82:0/1868742311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 msgr2=0x7fa8780ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.735+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/1868742311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa8780ff930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009990 tx=0x7fa86c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 -- 172.21.15.82:0/1868742311 shutdown_connections 2023-07-14T19:28:57.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/1868742311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 0x7fa87810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/1868742311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 0x7fa878104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/1868742311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa8780ff930 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 -- 172.21.15.82:0/1868742311 >> 172.21.15.82:0/1868742311 conn(0x7fa878075190 msgr2=0x7fa878075590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 -- 172.21.15.82:0/1868742311 shutdown_connections 2023-07-14T19:28:57.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 -- 172.21.15.82:0/1868742311 wait complete. 2023-07-14T19:28:57.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 Processor -- start 2023-07-14T19:28:57.739 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.736+0000 7fa87d9f4700 1 -- start start 2023-07-14T19:28:57.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa878196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 0x7fa878197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 0x7fa8781a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.740 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878197a60 con 0x7fa8780ff4e0 2023-07-14T19:28:57.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781a21d0 con 0x7fa878104b50 2023-07-14T19:28:57.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8781a2340 con 0x7fa8780ffe70 2023-07-14T19:28:57.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa878196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.741 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa878196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43060/0 (socket says 172.21.15.82:43060) 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 -- 172.21.15.82:0/861502366 learned_addr learned my addr 172.21.15.82:0/861502366 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa8767fc700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 0x7fa878197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa8777fe700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 0x7fa8781a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 -- 172.21.15.82:0/861502366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 msgr2=0x7fa878197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 0x7fa878197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 -- 172.21.15.82:0/861502366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 msgr2=0x7fa8781a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.742 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 0x7fa8781a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 -- 172.21.15.82:0/861502366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0095f0 con 0x7fa8780ff4e0 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa8767fc700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 0x7fa878197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa8777fe700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 0x7fa8781a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa876ffd700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa878196c10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa86800e9a0 tx=0x7fa86800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87c9f2700 1 -- 172.21.15.82:0/861502366 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868009790 con 0x7fa8780ff4e0 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8781a2620 con 0x7fa8780ff4e0 2023-07-14T19:28:57.743 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87c9f2700 1 -- 172.21.15.82:0/861502366 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86800cbd0 con 0x7fa8780ff4e0 2023-07-14T19:28:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87c9f2700 1 -- 172.21.15.82:0/861502366 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868018680 con 0x7fa8780ff4e0 2023-07-14T19:28:57.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.737+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8781a2b80 con 0x7fa8780ff4e0 2023-07-14T19:28:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.740+0000 7fa87c9f2700 1 -- 172.21.15.82:0/861502366 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fa86800cd40 con 0x7fa8780ff4e0 2023-07-14T19:28:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.740+0000 7fa87c9f2700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8600a1350 0x7fa8600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.749 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.740+0000 7fa87c9f2700 1 -- 172.21.15.82:0/861502366 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fa868014070 con 0x7fa8780ff4e0 2023-07-14T19:28:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.740+0000 7fa8767fc700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8600a1350 0x7fa8600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa8767fc700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8600a1350 0x7fa8600a3800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00b2e0 tx=0x7fa86c005fb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 Processor -- start 2023-07-14T19:28:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 -- start start 2023-07-14T19:28:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.750 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227dea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227deb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227ceacb0 con 0x55b227de6790 2023-07-14T19:28:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227d3da40 con 0x55b227deac30 2023-07-14T19:28:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227d36ed0 con 0x55b227de5e80 2023-07-14T19:28:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa8767fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227dea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.751 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa8767fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227dea6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43070/0 (socket says 172.21.15.82:43070) 2023-07-14T19:28:57.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa8767fc700 1 -- 172.21.15.82:0/2576487455 learned_addr learned my addr 172.21.15.82:0/2576487455 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.741+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227deb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa8767fc700 1 -- 172.21.15.82:0/2576487455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 msgr2=0x55b227de6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.752 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa8767fc700 1 -- 172.21.15.82:0/2576487455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 msgr2=0x55b227deb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227deb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa8767fc700 1 -- 172.21.15.82:0/2576487455 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b227b64de0 con 0x55b227de6790 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227dea6f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa86c015000 tx=0x7fa86c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc050bb46e769452 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa85dffb700 1 -- 172.21.15.82:0/2576487455 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c02b070 con 0x55b227de6790 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa85dffb700 1 -- 172.21.15.82:0/2576487455 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86c01e040 con 0x55b227de6790 2023-07-14T19:28:57.753 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.742+0000 7fa85dffb700 1 -- 172.21.15.82:0/2576487455 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c024930 con 0x55b227de6790 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2576487455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 msgr2=0x55b227dea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227dea6f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa86c015000 tx=0x7fa86c01f000 comp rx=0 tx=0).stop 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2576487455 shutdown_connections 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227deb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2576487455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227dea6f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.754 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2576487455 >> 172.21.15.82:0/2576487455 conn(0x55b227ddb680 msgr2=0x55b227de0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2576487455 shutdown_connections 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2576487455 wait complete. 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 -- start start 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 Processor -- start 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de5e80 0x55b227def770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.743+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de6790 0x55b227defcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.755 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227df01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227cd6ac0 con 0x55b227deac30 2023-07-14T19:28:57.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227cd67c0 con 0x55b227de5e80 2023-07-14T19:28:57.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227bba920 con 0x55b227de6790 2023-07-14T19:28:57.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa889b8cb80 10 client.0 _ll_register_callbacks cb 0x55b227cd4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:57.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227df01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.756 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227df01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43078/0 (socket says 172.21.15.82:43078) 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 -- 172.21.15.82:0/2743408050 learned_addr learned my addr 172.21.15.82:0/2743408050 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de5e80 0x55b227def770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de6790 0x55b227defcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 -- 172.21.15.82:0/2743408050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de6790 msgr2=0x55b227defcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de6790 0x55b227defcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 -- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de5e80 msgr2=0x55b227def770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de5e80 0x55b227def770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.757 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 -- 172.21.15.82:0/2743408050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87810c1f0 con 0x55b227deac30 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227df01f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa87000ce90 tx=0x7fa87000cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbf45fc701b3219 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa856ffd700 1 -- 172.21.15.82:0/2743408050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870011050 con 0x55b227deac30 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa856ffd700 1 -- 172.21.15.82:0/2743408050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fa870006e40 con 0x55b227deac30 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.744+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b227c904f0 con 0x55b227deac30 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa856ffd700 1 -- 172.21.15.82:0/2743408050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa870009ea0 con 0x55b227deac30 2023-07-14T19:28:57.758 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 10 client.15591 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b227b4dc60 con 0x55b227deac30 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa856ffd700 1 -- 172.21.15.82:0/2743408050 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b227c904f0 con 0x55b227deac30 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa856ffd700 1 -- 172.21.15.82:0/2743408050 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa870020950 con 0x55b227deac30 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 10 client.15591 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 20 client.15591 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227dffc00 0x55b227e02010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.759 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 4 client.15591 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b227db4da0 con 0x55b227dffc00 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.745+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227dffc00 0x55b227e02010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa856ffd700 10 client.15591 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227dffc00 0x55b227e02010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7b03de9a8a9847 server_cookie=d272b44d8772017 in_seq=0 out_seq=0 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa856ffd700 1 -- 172.21.15.82:0/2743408050 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55b227db4da0 con 0x55b227dffc00 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa856ffd700 10 client.15591 handle_command_reply: tid=0 2023-07-14T19:28:57.760 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 client.15591 shutdown 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 msgr2=0x55b227df01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227df01f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa87000ce90 tx=0x7fa87000cec0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 shutdown_connections 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227df01f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de5e80 0x55b227def770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227dffc00 0x55b227e02010 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2743408050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de6790 0x55b227defcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa856ffd700 0 client.15591 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.746+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 >> 172.21.15.82:0/2743408050 conn(0x55b227ddb680 msgr2=0x55b227de0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.747+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 shutdown_connections 2023-07-14T19:28:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.747+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2743408050 wait complete. 2023-07-14T19:28:57.762 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.747+0000 7fa889b8cb80 20 client.15591 trim_cache size 0 max 16384 2023-07-14T19:28:57.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 Processor -- start 2023-07-14T19:28:57.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 -- start start 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de6790 0x55b227df6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227defd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227de26d0 con 0x55b227deac30 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227de23b0 con 0x55b227de6790 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227e06dd0 con 0x55b227de5e80 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8767fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de6790 0x55b227df6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227defd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8767fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de6790 0x55b227df6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:39432/0 (socket says 172.21.15.82:39432) 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8767fc700 1 -- 172.21.15.82:0/2574012544 learned_addr learned my addr 172.21.15.82:0/2574012544 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8777fe700 1 -- 172.21.15.82:0/2574012544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 msgr2=0x55b227de8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8777fe700 1 -- 172.21.15.82:0/2574012544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de6790 msgr2=0x55b227df6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de6790 0x55b227df6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa8777fe700 1 -- 172.21.15.82:0/2574012544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b227ddf920 con 0x55b227deac30 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.762+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.763+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227defd00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa870004c20 tx=0x7fa870004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c246a5214eeef64e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.763+0000 7fa85e7fc700 1 -- 172.21.15.82:0/2574012544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870011040 con 0x55b227deac30 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.763+0000 7fa85e7fc700 1 -- 172.21.15.82:0/2574012544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa870012040 con 0x55b227deac30 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.763+0000 7fa85e7fc700 1 -- 172.21.15.82:0/2574012544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870007980 con 0x55b227deac30 2023-07-14T19:28:57.767 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.763+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2574012544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 msgr2=0x55b227defd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227defd00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa870004c20 tx=0x7fa870004a50 comp rx=0 tx=0).stop 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2574012544 shutdown_connections 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227deac30 0x55b227defd00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227de8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2574012544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227de6790 0x55b227df6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2574012544 >> 172.21.15.82:0/2574012544 conn(0x55b227ddb680 msgr2=0x55b227de8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2574012544 shutdown_connections 2023-07-14T19:28:57.768 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2574012544 wait complete. 2023-07-14T19:28:57.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- start start 2023-07-14T19:28:57.769 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 Processor -- start 2023-07-14T19:28:57.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227e479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227e47f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227e52870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.772 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227e52db0 con 0x55b227de6790 2023-07-14T19:28:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227e52f20 con 0x55b227deac30 2023-07-14T19:28:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.764+0000 7fa889b8cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b227e530e0 con 0x55b227de5e80 2023-07-14T19:28:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa889b8cb80 10 client.0 _ll_register_callbacks cb 0x7fa87000c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227e47f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.773 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227e47f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43090/0 (socket says 172.21.15.82:43090) 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 -- 172.21.15.82:0/2382730205 learned_addr learned my addr 172.21.15.82:0/2382730205 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8777fe700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227e52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227e479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 -- 172.21.15.82:0/2382730205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 msgr2=0x55b227e479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227e479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 -- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 msgr2=0x55b227e52870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227e52870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.774 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 -- 172.21.15.82:0/2382730205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b227b4dc60 con 0x55b227de6790 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa8767fc700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227e47f20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa86c031040 tx=0x7fa86c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25be79e561a7795 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa856ffd700 1 -- 172.21.15.82:0/2382730205 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c02b070 con 0x55b227de6790 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa86c01e040 con 0x55b227de6790 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa856ffd700 1 -- 172.21.15.82:0/2382730205 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fa86c024690 con 0x55b227de6790 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa856ffd700 1 -- 172.21.15.82:0/2382730205 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa86c024850 con 0x55b227de6790 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa889b8cb80 10 client.15597 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:57.775 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.765+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b227b8b250 con 0x55b227de6790 2023-07-14T19:28:57.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa856ffd700 1 -- 172.21.15.82:0/2382730205 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa86c049080 con 0x55b227de6790 2023-07-14T19:28:57.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa856ffd700 1 -- 172.21.15.82:0/2382730205 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa86c037070 con 0x55b227de6790 2023-07-14T19:28:57.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa889b8cb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:57.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa889b8cb80 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:57.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa889b8cb80 20 client.15597 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:57.776 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227e54530 0x55b227e56940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa889b8cb80 4 client.15597 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b227d2feb0 con 0x55b227e54530 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227e54530 0x55b227e56940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa856ffd700 10 client.15597 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.766+0000 7fa876ffd700 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227e54530 0x55b227e56940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a734fc8a72c9d6 server_cookie=3fe5201eb7abeacd in_seq=0 out_seq=0 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa856ffd700 1 -- 172.21.15.82:0/2382730205 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b227d2feb0 con 0x55b227e54530 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa856ffd700 10 client.15597 handle_command_reply: tid=0 2023-07-14T19:28:57.777 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 client.15597 shutdown 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 msgr2=0x55b227e47f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227e47f20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa86c031040 tx=0x7fa86c009310 comp rx=0 tx=0).stop 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 shutdown_connections 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b227deac30 0x55b227e52870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55b227e54530 0x55b227e56940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b227de5e80 0x55b227e479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.778 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 --2- 172.21.15.82:0/2382730205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b227de6790 0x55b227e47f20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa856ffd700 0 client.15597 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:57.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 >> 172.21.15.82:0/2382730205 conn(0x55b227ddb680 msgr2=0x55b227defd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 shutdown_connections 2023-07-14T19:28:57.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 1 -- 172.21.15.82:0/2382730205 wait complete. 2023-07-14T19:28:57.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa889b8cb80 20 client.15597 trim_cache size 0 max 16384 2023-07-14T19:28:57.779 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-07-14T19:28:57.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8600a1350 msgr2=0x7fa8600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8600a1350 0x7fa8600a3800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00b2e0 tx=0x7fa86c005fb0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 msgr2=0x7fa878196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:57.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa878196c10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa86800e9a0 tx=0x7fa86800ed60 comp rx=0 tx=0).stop 2023-07-14T19:28:57.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 shutdown_connections 2023-07-14T19:28:57.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8600a1350 0x7fa8600a3800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa878104b50 0x7fa8781a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.767+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8780ff4e0 0x7fa878196c10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.768+0000 7fa87d9f4700 1 --2- 172.21.15.82:0/861502366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8780ffe70 0x7fa878197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:57.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.768+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 >> 172.21.15.82:0/861502366 conn(0x7fa878075190 msgr2=0x7fa8780fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:57.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.768+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 shutdown_connections 2023-07-14T19:28:57.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:57.768+0000 7fa87d9f4700 1 -- 172.21.15.82:0/861502366 wait complete. 2023-07-14T19:28:57.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:57 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:28:57.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:57 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/119238069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:57.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:57 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:28:57.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:57 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/119238069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:58.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:57 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:28:58.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:57 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/119238069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:28:58.783 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-14T19:28:58.783 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-14T19:28:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 -- 172.21.15.82:0/1967825031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720100850 msgr2=0x7f1720104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 --2- 172.21.15.82:0/1967825031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720100850 0x7f1720104810 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f17100099c0 tx=0x7f1710009cd0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 -- 172.21.15.82:0/1967825031 shutdown_connections 2023-07-14T19:28:58.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 --2- 172.21.15.82:0/1967825031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720104ef0 0x7f172010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 --2- 172.21.15.82:0/1967825031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17200ffeb0 0x7f1720100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 --2- 172.21.15.82:0/1967825031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720100850 0x7f1720104810 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.909+0000 7f1727bef700 1 -- 172.21.15.82:0/1967825031 >> 172.21.15.82:0/1967825031 conn(0x7f17200fb7c0 msgr2=0x7f17200fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:58.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 -- 172.21.15.82:0/1967825031 shutdown_connections 2023-07-14T19:28:58.911 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 -- 172.21.15.82:0/1967825031 wait complete. 2023-07-14T19:28:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 Processor -- start 2023-07-14T19:28:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 -- start start 2023-07-14T19:28:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17200ffeb0 0x7f1720195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720100850 0x7f17201962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 0x7f172019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172019ad80 con 0x7f1720104ef0 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172019aef0 con 0x7f17200ffeb0 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f1727bef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172019b0b0 con 0x7f1720100850 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 0x7f172019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 0x7f172019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43104/0 (socket says 172.21.15.82:43104) 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.910+0000 7f172618c700 1 -- 172.21.15.82:0/511372592 learned_addr learned my addr 172.21.15.82:0/511372592 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:58.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172518a700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720100850 0x7f17201962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172598b700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17200ffeb0 0x7f1720195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172618c700 1 -- 172.21.15.82:0/511372592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720100850 msgr2=0x7f17201962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172618c700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720100850 0x7f17201962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172618c700 1 -- 172.21.15.82:0/511372592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17200ffeb0 msgr2=0x7f1720195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172618c700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17200ffeb0 0x7f1720195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172618c700 1 -- 172.21.15.82:0/511372592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100095f0 con 0x7f1720104ef0 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172518a700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720100850 0x7f17201962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:58.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f172618c700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 0x7f172019a740 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f171800ada0 tx=0x7f171800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:58.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f1716ffd700 1 -- 172.21.15.82:0/511372592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171800f070 con 0x7f1720104ef0 2023-07-14T19:28:58.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f1716ffd700 1 -- 172.21.15.82:0/511372592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1718007b30 con 0x7f1720104ef0 2023-07-14T19:28:58.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172019b390 con 0x7f1720104ef0 2023-07-14T19:28:58.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f1716ffd700 1 -- 172.21.15.82:0/511372592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718005440 con 0x7f1720104ef0 2023-07-14T19:28:58.916 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.911+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17201a2bd0 con 0x7f1720104ef0 2023-07-14T19:28:58.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.914+0000 7f1716ffd700 1 -- 172.21.15.82:0/511372592 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f1718004ad0 con 0x7f1720104ef0 2023-07-14T19:28:58.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.914+0000 7f1716ffd700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f170c0a1350 0x7f170c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.914+0000 7f172598b700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f170c0a1350 0x7f170c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1716ffd700 1 -- 172.21.15.82:0/511372592 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f17180cc430 con 0x7f1720104ef0 2023-07-14T19:28:58.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 Processor -- start 2023-07-14T19:28:58.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 -- start start 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f172598b700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f170c0a1350 0x7f170c0a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c005f60 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c84850d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84851250 0x555c848516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84865d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c84755d30 con 0x555c84863970 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c847a8ac0 con 0x555c84854ba0 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f1733d87b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c847a1f50 con 0x555c84851250 2023-07-14T19:28:58.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84865d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84865d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43112/0 (socket says 172.21.15.82:43112) 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f172618c700 1 -- 172.21.15.82:0/1367442694 learned_addr learned my addr 172.21.15.82:0/1367442694 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.915+0000 7f172518a700 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84851250 0x555c848516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172598b700 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c84850d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172618c700 1 -- 172.21.15.82:0/1367442694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84851250 msgr2=0x555c848516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172618c700 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84851250 0x555c848516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172618c700 1 -- 172.21.15.82:0/1367442694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 msgr2=0x555c84850d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172618c700 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c84850d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172618c700 1 -- 172.21.15.82:0/1367442694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c845cfe60 con 0x555c84863970 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f172618c700 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84865d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17180991b0 tx=0x7f1718099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74a7c41d9211e67 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f170bfff700 1 -- 172.21.15.82:0/1367442694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171809d4e0 con 0x555c84863970 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f170bfff700 1 -- 172.21.15.82:0/1367442694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17180a1870 con 0x555c84863970 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.916+0000 7f170bfff700 1 -- 172.21.15.82:0/1367442694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171809f110 con 0x555c84863970 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 -- 172.21.15.82:0/1367442694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 msgr2=0x555c84865d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84865d50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17180991b0 tx=0x7f1718099300 comp rx=0 tx=0).stop 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 -- 172.21.15.82:0/1367442694 shutdown_connections 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84865d50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c84850d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 --2- 172.21.15.82:0/1367442694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84851250 0x555c848516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 -- 172.21.15.82:0/1367442694 >> 172.21.15.82:0/1367442694 conn(0x555c84846700 msgr2=0x555c8484bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 -- 172.21.15.82:0/1367442694 shutdown_connections 2023-07-14T19:28:58.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 -- 172.21.15.82:0/1367442694 wait complete. 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 -- start start 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.917+0000 7f1733d87b80 1 Processor -- start 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84863360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84860c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c84741b40 con 0x555c84863970 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c84741840 con 0x555c84851250 2023-07-14T19:28:58.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c846259a0 con 0x555c84854ba0 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f1733d87b80 10 client.0 _ll_register_callbacks cb 0x555c8473f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84860c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172518a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84860c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43118/0 (socket says 172.21.15.82:43118) 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 -- 172.21.15.82:0/3971732991 learned_addr learned my addr 172.21.15.82:0/3971732991 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172598b700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84863360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 -- 172.21.15.82:0/3971732991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 msgr2=0x555c848605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 -- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 msgr2=0x555c84863360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84863360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 -- 172.21.15.82:0/3971732991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17201a2bd0 con 0x555c84863970 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172518a700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f172618c700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84860c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17180c9c30 tx=0x7f1718099570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f015010a26efef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f17097fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:58.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.918+0000 7f17097fa700 1 -- 172.21.15.82:0/3971732991 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180c9e00 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f17097fa700 1 -- 172.21.15.82:0/3971732991 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f17180c9f60 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c846fb570 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f17097fa700 1 -- 172.21.15.82:0/3971732991 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17180c7020 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 10 client.15609 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c845b8ce0 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f17097fa700 1 -- 172.21.15.82:0/3971732991 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c846fb570 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f17097fa700 1 -- 172.21.15.82:0/3971732991 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f17180989a0 con 0x555c84863970 2023-07-14T19:28:58.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 10 client.15609 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 20 client.15609 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c84873020 0x555c84875430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 4 client.15609 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c8481fe20 con 0x555c84873020 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.919+0000 7f172598b700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c84873020 0x555c84875430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.920+0000 7f17097fa700 10 client.15609 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:58.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.920+0000 7f172598b700 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c84873020 0x555c84875430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df55d3bafa2b5aa1 server_cookie=c1f5430e1238e37d in_seq=0 out_seq=0 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.920+0000 7f17097fa700 1 -- 172.21.15.82:0/3971732991 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x555c8481fe20 con 0x555c84873020 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.920+0000 7f17097fa700 10 client.15609 handle_command_reply: tid=0 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.920+0000 7f1733d87b80 1 client.15609 shutdown 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 msgr2=0x555c84860c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84860c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17180c9c30 tx=0x7f1718099570 comp rx=0 tx=0).stop 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 shutdown_connections 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c84860c60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84863360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c84873020 0x555c84875430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 --2- 172.21.15.82:0/3971732991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f17097fa700 0 client.15609 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 >> 172.21.15.82:0/3971732991 conn(0x555c84846700 msgr2=0x555c8484bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 shutdown_connections 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 1 -- 172.21.15.82:0/3971732991 wait complete. 2023-07-14T19:28:58.933 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.921+0000 7f1733d87b80 20 client.15609 trim_cache size 0 max 16384 2023-07-14T19:28:58.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 Processor -- start 2023-07-14T19:28:58.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 -- start start 2023-07-14T19:28:58.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 0x555c8486a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c8485fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84863970 0x555c848605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c8484d600 con 0x555c84851250 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c84851fa0 con 0x555c84854ba0 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f1733d87b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c84854f70 con 0x555c84863970 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f172598b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 0x555c8486a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f172598b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 0x555c8486a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43122/0 (socket says 172.21.15.82:43122) 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f172598b700 1 -- 172.21.15.82:0/675977857 learned_addr learned my addr 172.21.15.82:0/675977857 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f172518a700 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c8485fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f172598b700 1 -- 172.21.15.82:0/675977857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84863970 msgr2=0x555c848605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.940+0000 7f172618c700 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84863970 0x555c848605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f172598b700 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84863970 0x555c848605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f172598b700 1 -- 172.21.15.82:0/675977857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 msgr2=0x555c8485fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f172598b700 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c8485fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f172598b700 1 -- 172.21.15.82:0/675977857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c845f62d0 con 0x555c84851250 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f172598b700 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 0x555c8486a5c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b7c0 tx=0x7f171c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c5aa120f2b1cbc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:58.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f1714daa700 1 -- 172.21.15.82:0/675977857 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c025040 con 0x555c84851250 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f1714daa700 1 -- 172.21.15.82:0/675977857 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c0272b0 con 0x555c84851250 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.941+0000 7f1714daa700 1 -- 172.21.15.82:0/675977857 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c01f430 con 0x555c84851250 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 -- 172.21.15.82:0/675977857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 msgr2=0x555c8486a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 0x555c8486a5c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b7c0 tx=0x7f171c012000 comp rx=0 tx=0).stop 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 -- 172.21.15.82:0/675977857 shutdown_connections 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84863970 0x555c848605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84851250 0x555c8486a5c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- 172.21.15.82:0/675977857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84854ba0 0x555c8485fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 -- 172.21.15.82:0/675977857 >> 172.21.15.82:0/675977857 conn(0x555c84846700 msgr2=0x555c8484f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 -- 172.21.15.82:0/675977857 shutdown_connections 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 -- 172.21.15.82:0/675977857 wait complete. 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 -- start start 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 Processor -- start 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84868b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.942+0000 7f1733d87b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c848695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f1733d87b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c848c6d70 con 0x555c84863970 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f1733d87b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c848c6eb0 con 0x555c84851250 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f1733d87b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c848c7070 con 0x555c84854ba0 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f1733d87b80 10 client.0 _ll_register_callbacks cb 0x7f1710009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c848695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c848695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43130/0 (socket says 172.21.15.82:43130) 2023-07-14T19:28:58.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 -- 172.21.15.82:0/2460921440 learned_addr learned my addr 172.21.15.82:0/2460921440 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172598b700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84868b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172518a700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 -- 172.21.15.82:0/2460921440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 msgr2=0x555c848690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 -- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 msgr2=0x555c84868b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84868b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 -- 172.21.15.82:0/2460921440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c00f970 con 0x555c84863970 2023-07-14T19:28:58.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f172618c700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c848695f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f171800c730 tx=0x7f1718099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240e00159628dd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:58.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f17097fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:58.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f17097fa700 1 -- 172.21.15.82:0/2460921440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718099710 con 0x555c84863970 2023-07-14T19:28:58.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f17097fa700 1 -- 172.21.15.82:0/2460921440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f17180194c0 con 0x555c84863970 2023-07-14T19:28:58.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17180a1870 con 0x555c84863970 2023-07-14T19:28:58.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f17097fa700 1 -- 172.21.15.82:0/2460921440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1718099870 con 0x555c84863970 2023-07-14T19:28:58.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.943+0000 7f1733d87b80 10 client.15615 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c845ce330 con 0x555c84863970 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f17097fa700 1 -- 172.21.15.82:0/2460921440 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17180a1870 con 0x555c84863970 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f17097fa700 1 -- 172.21.15.82:0/2460921440 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f17180c9e00 con 0x555c84863970 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 20 client.15615 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c848c8350 0x555c848ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:58.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 4 client.15615 mds_command: new command op to 24524 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-14T19:28:58.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555c8479af30 con 0x555c848c8350 2023-07-14T19:28:58.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.944+0000 7f172598b700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c848c8350 0x555c848ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:58.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.945+0000 7f17097fa700 10 client.15615 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:58.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.945+0000 7f172598b700 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c848c8350 0x555c848ca730 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a215456fbcc38c9 server_cookie=1719e3f9ed252d9d in_seq=0 out_seq=0 2023-07-14T19:28:58.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f17097fa700 1 -- 172.21.15.82:0/2460921440 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c8479af30 con 0x555c848c8350 2023-07-14T19:28:58.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f17097fa700 10 client.15615 handle_command_reply: tid=0 2023-07-14T19:28:58.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 client.15615 shutdown 2023-07-14T19:28:58.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 msgr2=0x555c848695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c848695f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f171800c730 tx=0x7f1718099280 comp rx=0 tx=0).stop 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 shutdown_connections 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c84863970 0x555c848695f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c84851250 0x555c84868b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x555c848c8350 0x555c848ca730 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 --2- 172.21.15.82:0/2460921440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c84854ba0 0x555c848690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 >> 172.21.15.82:0/2460921440 conn(0x555c84846700 msgr2=0x555c8484b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f17097fa700 0 client.15615 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 shutdown_connections 2023-07-14T19:28:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 1 -- 172.21.15.82:0/2460921440 wait complete. 2023-07-14T19:28:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.946+0000 7f1733d87b80 20 client.15615 trim_cache size 0 max 16384 2023-07-14T19:28:58.958 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:58.958 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2023-07-14T19:28:58.959 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "96cf0cc1-2bb7-48a1-be95-375642a0df3f", 2023-07-14T19:28:58.959 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2023-07-14T19:28:58.959 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f170c0a1350 msgr2=0x7f170c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f170c0a1350 0x7f170c0a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c005f60 comp rx=0 tx=0).stop 2023-07-14T19:28:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 msgr2=0x7f172019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 0x7f172019a740 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f171800ada0 tx=0x7f171800c330 comp rx=0 tx=0).stop 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 shutdown_connections 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f170c0a1350 0x7f170c0a3800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1720104ef0 0x7f172019a740 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17200ffeb0 0x7f1720195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 --2- 172.21.15.82:0/511372592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1720100850 0x7f17201962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 >> 172.21.15.82:0/511372592 conn(0x7f17200fb7c0 msgr2=0x7f17200fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:58.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 shutdown_connections 2023-07-14T19:28:58.962 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:58.947+0000 7f1727bef700 1 -- 172.21.15.82:0/511372592 wait complete. 2023-07-14T19:28:58.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T19:28:58.985 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:58 smithi082 ceph-mon[127461]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2023-07-14T19:28:58.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:58 smithi161 ceph-mon[129184]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2023-07-14T19:28:59.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 -- 172.21.15.82:0/3030461739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5680ffeb0 msgr2=0x7ff568100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/3030461739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5680ffeb0 0x7ff568100280 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff5580099c0 tx=0x7ff558009cd0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 -- 172.21.15.82:0/3030461739 shutdown_connections 2023-07-14T19:28:59.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/3030461739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 0x7ff56810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/3030461739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5680ffeb0 0x7ff568100280 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/3030461739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff568100850 0x7ff568104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 -- 172.21.15.82:0/3030461739 >> 172.21.15.82:0/3030461739 conn(0x7ff5680fb7c0 msgr2=0x7ff5680fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 -- 172.21.15.82:0/3030461739 shutdown_connections 2023-07-14T19:28:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.088+0000 7ff56e0ab700 1 -- 172.21.15.82:0/3030461739 wait complete. 2023-07-14T19:28:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 Processor -- start 2023-07-14T19:28:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 -- start start 2023-07-14T19:28:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5680ffeb0 0x7ff568195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 0x7ff568196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 0x7ff56819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56819adf0 con 0x7ff568100850 2023-07-14T19:28:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56819af60 con 0x7ff5680ffeb0 2023-07-14T19:28:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff56e0ab700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56819b120 con 0x7ff568104ef0 2023-07-14T19:28:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 0x7ff568196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 0x7ff568196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43142/0 (socket says 172.21.15.82:43142) 2023-07-14T19:28:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 -- 172.21.15.82:0/2546067456 learned_addr learned my addr 172.21.15.82:0/2546067456 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff567fff700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 0x7ff56819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff5677fe700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5680ffeb0 0x7ff568195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 -- 172.21.15.82:0/2546067456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 msgr2=0x7ff56819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 0x7ff56819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 -- 172.21.15.82:0/2546067456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5680ffeb0 msgr2=0x7ff568195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5680ffeb0 0x7ff568195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff566ffd700 1 -- 172.21.15.82:0/2546067456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580095f0 con 0x7ff568100850 2023-07-14T19:28:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.089+0000 7ff567fff700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 0x7ff56819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.090+0000 7ff566ffd700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 0x7ff568196350 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00c8c0 tx=0x7ff55c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.090+0000 7ff564ff9700 1 -- 172.21.15.82:0/2546067456 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c004220 con 0x7ff568100850 2023-07-14T19:28:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.090+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56819b400 con 0x7ff568100850 2023-07-14T19:28:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.090+0000 7ff564ff9700 1 -- 172.21.15.82:0/2546067456 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c004cf0 con 0x7ff568100850 2023-07-14T19:28:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.090+0000 7ff564ff9700 1 -- 172.21.15.82:0/2546067456 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c005700 con 0x7ff568100850 2023-07-14T19:28:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.090+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5681a2c40 con 0x7ff568100850 2023-07-14T19:28:59.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.093+0000 7ff564ff9700 1 -- 172.21.15.82:0/2546067456 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 22) v1 ==== 135891+0+0 (secure 0 0 0) 0x7ff55c004380 con 0x7ff568100850 2023-07-14T19:28:59.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.093+0000 7ff564ff9700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5500a1350 0x7ff5500a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.093+0000 7ff57a243b80 1 Processor -- start 2023-07-14T19:28:59.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.093+0000 7ff5677fe700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5500a1350 0x7ff5500a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.093+0000 7ff564ff9700 1 -- 172.21.15.82:0/2546067456 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7ff55c0cc9e0 con 0x7ff568100850 2023-07-14T19:28:59.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 -- start start 2023-07-14T19:28:59.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559588129e80 0x55958812a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x55958812e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812ec30 0x55958812f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958802ecb0 con 0x55958812a790 2023-07-14T19:28:59.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559588081a40 con 0x55958812ec30 2023-07-14T19:28:59.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff57a243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958807aed0 con 0x559588129e80 2023-07-14T19:28:59.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x55958812e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x55958812e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43150/0 (socket says 172.21.15.82:43150) 2023-07-14T19:28:59.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 -- 172.21.15.82:0/4229458523 learned_addr learned my addr 172.21.15.82:0/4229458523 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:59.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff5677fe700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559588129e80 0x55958812a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff567fff700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812ec30 0x55958812f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 -- 172.21.15.82:0/4229458523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559588129e80 msgr2=0x55958812a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559588129e80 0x55958812a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 -- 172.21.15.82:0/4229458523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812ec30 msgr2=0x55958812f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812ec30 0x55958812f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 -- 172.21.15.82:0/4229458523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559587ea8de0 con 0x55958812a790 2023-07-14T19:28:59.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff567fff700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812ec30 0x55958812f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff5677fe700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559588129e80 0x55958812a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff5677fe700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5500a1350 0x7ff5500a3800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c00 tx=0x7ff5580051f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff566ffd700 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x55958812e6f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff55c096440 tx=0x7ff55c0993b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74eb245d9f961bb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.094+0000 7ff54dffb700 1 -- 172.21.15.82:0/4229458523 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a1e20 con 0x55958812a790 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff54dffb700 1 -- 172.21.15.82:0/4229458523 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c097d70 con 0x55958812a790 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff54dffb700 1 -- 172.21.15.82:0/4229458523 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c099e90 con 0x55958812a790 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 -- 172.21.15.82:0/4229458523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 msgr2=0x55958812e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x55958812e6f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff55c096440 tx=0x7ff55c0993b0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 -- 172.21.15.82:0/4229458523 shutdown_connections 2023-07-14T19:28:59.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812ec30 0x55958812f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559588129e80 0x55958812a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 --2- 172.21.15.82:0/4229458523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x55958812e6f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 -- 172.21.15.82:0/4229458523 >> 172.21.15.82:0/4229458523 conn(0x55958811f680 msgr2=0x559588124ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:59.111 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 -- 172.21.15.82:0/4229458523 shutdown_connections 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.095+0000 7ff57a243b80 1 -- 172.21.15.82:0/4229458523 wait complete. 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 -- start start 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 Processor -- start 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 0x559588136450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812a790 0x5595881336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588133e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.112 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958801aac0 con 0x559588129e80 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958801a7c0 con 0x55958812a790 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559587efe920 con 0x55958812ec30 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff57a243b80 10 client.0 _ll_register_callbacks cb 0x559588018760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 0x559588136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 0x559588136450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43154/0 (socket says 172.21.15.82:43154) 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff5677fe700 1 -- 172.21.15.82:0/1916112047 learned_addr learned my addr 172.21.15.82:0/1916112047 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.096+0000 7ff566ffd700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812a790 0x5595881336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff567fff700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588133e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff5677fe700 1 -- 172.21.15.82:0/1916112047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 msgr2=0x559588133e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff5677fe700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588133e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff5677fe700 1 -- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812a790 msgr2=0x5595881336d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff5677fe700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812a790 0x5595881336d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff5677fe700 1 -- 172.21.15.82:0/1916112047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5681a2c40 con 0x559588129e80 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff5677fe700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 0x559588136450 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff558015000 tx=0x7ff558009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305f93fc645bb168 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:59.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff546ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff546ffd700 1 -- 172.21.15.82:0/1916112047 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558005680 con 0x559588129e80 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff546ffd700 1 -- 172.21.15.82:0/1916112047 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7ff55802c040 con 0x559588129e80 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559587fd44f0 con 0x559588129e80 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff546ffd700 1 -- 172.21.15.82:0/1916112047 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff558003d50 con 0x559588129e80 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff57a243b80 10 client.15627 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559587e91c60 con 0x559588129e80 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff546ffd700 1 -- 172.21.15.82:0/1916112047 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559587fd44f0 con 0x559588129e80 2023-07-14T19:28:59.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.097+0000 7ff546ffd700 1 -- 172.21.15.82:0/1916112047 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff55800f460 con 0x559588129e80 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff57a243b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff57a243b80 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff57a243b80 20 client.15627 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff57a243b80 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x559588145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff57a243b80 4 client.15627 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595880f8da0 con 0x559588143ab0 2023-07-14T19:28:59.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff566ffd700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x559588145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff566ffd700 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x559588145ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c887ddcb7dc89d82 server_cookie=b3a49e832ab5a053 in_seq=0 out_seq=0 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.098+0000 7ff546ffd700 10 client.15627 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff546ffd700 1 -- 172.21.15.82:0/1916112047 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5595880f8da0 con 0x559588143ab0 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff546ffd700 10 client.15627 handle_command_reply: tid=0 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 client.15627 shutdown 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 msgr2=0x559588136450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 0x559588136450 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff558015000 tx=0x7ff558009cd0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 shutdown_connections 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x559588145ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588133e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559588129e80 0x559588136450 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff546ffd700 0 client.15627 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 --2- 172.21.15.82:0/1916112047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55958812a790 0x5595881336d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 >> 172.21.15.82:0/1916112047 conn(0x55958811f680 msgr2=0x559588124a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:59.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 shutdown_connections 2023-07-14T19:28:59.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 1 -- 172.21.15.82:0/1916112047 wait complete. 2023-07-14T19:28:59.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.099+0000 7ff57a243b80 20 client.15627 trim_cache size 0 max 16384 2023-07-14T19:28:59.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 Processor -- start 2023-07-14T19:28:59.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 -- start start 2023-07-14T19:28:59.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x55958812c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588135ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958812ad20 con 0x55958812a790 2023-07-14T19:28:59.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55958812af40 con 0x559588129e80 2023-07-14T19:28:59.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff57a243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559588134cb0 con 0x55958812ec30 2023-07-14T19:28:59.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588135ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588135ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43164/0 (socket says 172.21.15.82:43164) 2023-07-14T19:28:59.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.116+0000 7ff566ffd700 1 -- 172.21.15.82:0/2794176491 learned_addr learned my addr 172.21.15.82:0/2794176491 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:59.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff5677fe700 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x55958812c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff567fff700 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff566ffd700 1 -- 172.21.15.82:0/2794176491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 msgr2=0x559588132eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.128 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff566ffd700 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588132eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff566ffd700 1 -- 172.21.15.82:0/2794176491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 msgr2=0x55958812c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff566ffd700 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x55958812c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff566ffd700 1 -- 172.21.15.82:0/2794176491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559587ecf250 con 0x55958812a790 2023-07-14T19:28:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff566ffd700 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588135ff0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ca1e0 tx=0x7ff55c099b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3f051ea9d81fa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:59.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff54e7fc700 1 -- 172.21.15.82:0/2794176491 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c004e60 con 0x55958812a790 2023-07-14T19:28:59.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff54e7fc700 1 -- 172.21.15.82:0/2794176491 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c0ca3b0 con 0x55958812a790 2023-07-14T19:28:59.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.117+0000 7ff54e7fc700 1 -- 172.21.15.82:0/2794176491 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c099c30 con 0x55958812a790 2023-07-14T19:28:59.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 -- 172.21.15.82:0/2794176491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 msgr2=0x559588135ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.130 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588135ff0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ca1e0 tx=0x7ff55c099b50 comp rx=0 tx=0).stop 2023-07-14T19:28:59.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 -- 172.21.15.82:0/2794176491 shutdown_connections 2023-07-14T19:28:59.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x559588132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.131 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x55958812c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 --2- 172.21.15.82:0/2794176491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588135ff0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 -- 172.21.15.82:0/2794176491 >> 172.21.15.82:0/2794176491 conn(0x55958811f680 msgr2=0x55958812cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:59.132 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 -- 172.21.15.82:0/2794176491 shutdown_connections 2023-07-14T19:28:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 -- 172.21.15.82:0/2794176491 wait complete. 2023-07-14T19:28:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 -- start start 2023-07-14T19:28:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.118+0000 7ff57a243b80 1 Processor -- start 2023-07-14T19:28:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x5595881955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588195b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.133 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x55958818f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559588190030 con 0x55958812a790 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595881901a0 con 0x559588129e80 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559588190360 con 0x55958812ec30 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff57a243b80 10 client.0 _ll_register_callbacks cb 0x7ff5580054e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43170/0 (socket says 172.21.15.82:43170) 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 -- 172.21.15.82:0/945287553 learned_addr learned my addr 172.21.15.82:0/945287553 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:28:59.134 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff567fff700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x55958818f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff5677fe700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x5595881955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 -- 172.21.15.82:0/945287553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 msgr2=0x55958818f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x55958818f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 -- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 msgr2=0x5595881955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x5595881955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 -- 172.21.15.82:0/945287553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c0b1040 con 0x55958812a790 2023-07-14T19:28:59.135 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff5677fe700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x5595881955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:28:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff566ffd700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588195b30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ca190 tx=0x7ff55c005120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f97f99c66a99ecc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:28:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff546ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:28:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.119+0000 7ff546ffd700 1 -- 172.21.15.82:0/945287553 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c098fd0 con 0x55958812a790 2023-07-14T19:28:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff546ffd700 1 -- 172.21.15.82:0/945287553 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7ff55c099130 con 0x55958812a790 2023-07-14T19:28:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff55c097d70 con 0x55958812a790 2023-07-14T19:28:59.136 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff546ffd700 1 -- 172.21.15.82:0/945287553 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff55c099c30 con 0x55958812a790 2023-07-14T19:28:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 10 client.15633 fetch_fsmap learned FSMap version 24 2023-07-14T19:28:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559587ea72b0 con 0x55958812a790 2023-07-14T19:28:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff546ffd700 1 -- 172.21.15.82:0/945287553 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c097d70 con 0x55958812a790 2023-07-14T19:28:59.137 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff546ffd700 1 -- 172.21.15.82:0/945287553 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff55c0989a0 con 0x55958812a790 2023-07-14T19:28:59.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 24 2023-07-14T19:28:59.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:28:59.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 20 client.15633 populate_metadata read hostname 'smithi082' 2023-07-14T19:28:59.138 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x55958819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:28:59.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 4 client.15633 mds_command: new command op to 24524 tid=0[{"prefix": "scrub status"}] 2023-07-14T19:28:59.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559588073eb0 con 0x559588143ab0 2023-07-14T19:28:59.139 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.120+0000 7ff5677fe700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x55958819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:28:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff546ffd700 10 client.15633 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff5677fe700 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x55958819ef30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb39fc8dab032cd2 server_cookie=fa773c4f6a57f088 in_seq=0 out_seq=0 2023-07-14T19:28:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff546ffd700 1 -- 172.21.15.82:0/945287553 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559588073eb0 con 0x559588143ab0 2023-07-14T19:28:59.140 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff546ffd700 10 client.15633 handle_command_reply: tid=0 2023-07-14T19:28:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 client.15633 shutdown 2023-07-14T19:28:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 msgr2=0x559588195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.141 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588195b30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0ca190 tx=0x7ff55c005120 comp rx=0 tx=0).stop 2023-07-14T19:28:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 shutdown_connections 2023-07-14T19:28:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559588143ab0 0x55958819ef30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55958812ec30 0x55958818f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.142 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559588129e80 0x5595881955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff546ffd700 0 client.15633 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:28:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 --2- 172.21.15.82:0/945287553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55958812a790 0x559588195b30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.143 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 >> 172.21.15.82:0/945287553 conn(0x55958811f680 msgr2=0x55958812dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 shutdown_connections 2023-07-14T19:28:59.144 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 1 -- 172.21.15.82:0/945287553 wait complete. 2023-07-14T19:28:59.145 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.121+0000 7ff57a243b80 20 client.15633 trim_cache size 0 max 16384 2023-07-14T19:28:59.145 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:28:59.146 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-14T19:28:59.146 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2023-07-14T19:28:59.147 INFO:teuthology.orchestra.run.smithi082.stdout: "96cf0cc1-2bb7-48a1-be95-375642a0df3f": { 2023-07-14T19:28:59.147 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2023-07-14T19:28:59.147 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "96cf0cc1-2bb7-48a1-be95-375642a0df3f", 2023-07-14T19:28:59.148 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,force" 2023-07-14T19:28:59.148 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:28:59.148 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:28:59.148 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:28:59.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5500a1350 msgr2=0x7ff5500a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5500a1350 0x7ff5500a3800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c00 tx=0x7ff5580051f0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 msgr2=0x7ff568196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 0x7ff568196350 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00c8c0 tx=0x7ff55c00cbd0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 shutdown_connections 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5500a1350 0x7ff5500a3800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff568104ef0 0x7ff56819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5680ffeb0 0x7ff568195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 --2- 172.21.15.82:0/2546067456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff568100850 0x7ff568196350 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:28:59.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 >> 172.21.15.82:0/2546067456 conn(0x7ff5680fb7c0 msgr2=0x7ff5680fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:28:59.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 shutdown_connections 2023-07-14T19:28:59.151 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:28:59.122+0000 7ff56e0ab700 1 -- 172.21.15.82:0/2546067456 wait complete. 2023-07-14T19:28:59.151 INFO:tasks.cephfs.filesystem:scrub status for tag:96cf0cc1-2bb7-48a1-be95-375642a0df3f - {'path': '/', 'tag': '96cf0cc1-2bb7-48a1-be95-375642a0df3f', 'options': 'recursive,force'} 2023-07-14T19:28:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:58 smithi133 ceph-mon[128894]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2023-07-14T19:28:59.183 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:28:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: 2023-07-14T19:28:58.947+0000 7f9e3fbcd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [...2,head] /volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/ auth{2=1} v20 ap=1 f(v0 m2023-07-14T19:28:35.964556+0000 1=0+1) n(v0 rc2023-07-14T19:28:35.972555+0000 2=0+2) (inest mix) (iversion lock) caps={15252=pAsLsXsFs/pAsLsXsFsx@1} mcw={2=AsLsXsFs} | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55b97d51ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000001fb:[<0x100000001fa/6e5f72af-0c6b-494d-b56b-ad34fb765505 v18>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v27>,<0x1/volumes v23>]//[]","memoryvalue":"(2)0x100000001fb:[<0x100000001fa/6e5f72af-0c6b-494d-b56b-ad34fb765505 v20>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v27>,<0x1/volumes v18>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-14T19:28:35.964556+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-07-14T19:28:35.972555+0000 2=0+2)","memory_value.dirstat":"f(v0 m2023-07-14T19:28:35.964556+0000 1=0+1)","memory_value.rstat":"n(v0 rc2023-07-14T19:28:35.972555+0000 2=0+2)","error_str":""},"return_code":0} 2023-07-14T19:28:59.184 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:28:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: 2023-07-14T19:28:58.947+0000 7f9e3fbcd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /volumes/_nogroup/sv_1/.meta auth v14 ap=1 s=123 n(v0 rc2023-07-14T19:27:03.704301+0000 b123 1=1+0) (iversion lock) | replicated=0 authpin=1 scrubqueue=0 0x55b97d51cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001fc:[<0x100000001fa/.meta v14>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v27>,<0x1/volumes v23>]//[]","memoryvalue":"(3)0x100000001fc:[<0x100000001fa/.meta v14>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v27>,<0x1/volumes v18>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-07-14T19:28:59.184 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:28:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: 2023-07-14T19:28:58.947+0000 7f9e3fbcd700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/client.0/ auth v4 ap=1 f() n(v0 rc2023-07-14T19:28:35.972555+0000 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={15252=pAsLsXsFsx/AsLsXsFsx@2},l=15252 | ptrwaiter=0 lock=0 importingcaps=0 dirfrag=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55b97d51d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000005:[<0x100000001fb/client.0 v2>,<0x100000001fa/6e5f72af-0c6b-494d-b56b-ad34fb765505 v18>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v27>,<0x1/volumes v23>]//[]","memoryvalue":"(2)0x10000000005:[<0x100000001fb/client.0 v4>,<0x100000001fa/6e5f72af-0c6b-494d-b56b-ad34fb765505 v20>,<0x10000000001/sv_1 v41>,<0x10000000000/_nogroup v27>,<0x1/volumes v18>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 rc2023-07-14T19:28:35.972555+0000 1=0+1)","error_str":""},"return_code":0} 2023-07-14T19:28:59.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:59 smithi161 ceph-mon[129184]: scrub summary: active paths [/] 2023-07-14T19:28:59.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:59 smithi161 ceph-mon[129184]: Scrub error on inode 0x100000001fb (/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505) see mds.h log and `damage ls` output for details 2023-07-14T19:28:59.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:59 smithi161 ceph-mon[129184]: Scrub error on inode 0x100000001fc (/volumes/_nogroup/sv_1/.meta) see mds.h log and `damage ls` output for details 2023-07-14T19:28:59.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:59 smithi161 ceph-mon[129184]: Scrub error on inode 0x10000000005 (/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/client.0) see mds.h log and `damage ls` output for details 2023-07-14T19:28:59.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:28:59 smithi161 ceph-mon[129184]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:59 smithi133 ceph-mon[128894]: scrub summary: active paths [/] 2023-07-14T19:29:00.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:59 smithi133 ceph-mon[128894]: Scrub error on inode 0x100000001fb (/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505) see mds.h log and `damage ls` output for details 2023-07-14T19:29:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:59 smithi133 ceph-mon[128894]: Scrub error on inode 0x100000001fc (/volumes/_nogroup/sv_1/.meta) see mds.h log and `damage ls` output for details 2023-07-14T19:29:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:59 smithi133 ceph-mon[128894]: Scrub error on inode 0x10000000005 (/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/client.0) see mds.h log and `damage ls` output for details 2023-07-14T19:29:00.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:28:59 smithi133 ceph-mon[128894]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:59 smithi082 ceph-mon[127461]: scrub summary: active paths [/] 2023-07-14T19:29:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:59 smithi082 ceph-mon[127461]: Scrub error on inode 0x100000001fb (/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505) see mds.h log and `damage ls` output for details 2023-07-14T19:29:00.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:59 smithi082 ceph-mon[127461]: Scrub error on inode 0x100000001fc (/volumes/_nogroup/sv_1/.meta) see mds.h log and `damage ls` output for details 2023-07-14T19:29:00.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:59 smithi082 ceph-mon[127461]: Scrub error on inode 0x10000000005 (/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/client.0) see mds.h log and `damage ls` output for details 2023-07-14T19:29:00.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:28:59 smithi082 ceph-mon[127461]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:00.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:00 smithi161 ceph-mon[129184]: scrub queued for path: / 2023-07-14T19:29:00.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:00 smithi161 ceph-mon[129184]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:00.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:00 smithi161 ceph-mon[129184]: scrub summary: active paths [/] 2023-07-14T19:29:00.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:00 smithi161 ceph-mon[129184]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:00.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:00 smithi161 ceph-mon[129184]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2023-07-14T19:29:01.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:00 smithi082 ceph-mon[127461]: scrub queued for path: / 2023-07-14T19:29:01.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:00 smithi082 ceph-mon[127461]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:01.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:00 smithi082 ceph-mon[127461]: scrub summary: active paths [/] 2023-07-14T19:29:01.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:00 smithi082 ceph-mon[127461]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:01.013 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:00 smithi082 ceph-mon[127461]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2023-07-14T19:29:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:00 smithi133 ceph-mon[128894]: scrub queued for path: / 2023-07-14T19:29:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:00 smithi133 ceph-mon[128894]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:01.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:00 smithi133 ceph-mon[128894]: scrub summary: active paths [/] 2023-07-14T19:29:01.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:00 smithi133 ceph-mon[128894]: scrub summary: idle+waiting paths [/] 2023-07-14T19:29:01.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:00 smithi133 ceph-mon[128894]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2023-07-14T19:29:01.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:29:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:29:01.010+0000 7fac787e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:29:01.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:29:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:29:01.010+0000 7fac787e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:29:01.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:29:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:29:01.010+0000 7fac787e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:29:01.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:29:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:29:01.010+0000 7fac787e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:29:01.464 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:29:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:29:01.010+0000 7fac787e8700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-14T19:29:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 -- 172.21.15.82:0/2296894536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33780ff390 msgr2=0x7f33780ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 --2- 172.21.15.82:0/2296894536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33780ff390 0x7f33780ff800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3364009990 tx=0x7f3364009ca0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 -- 172.21.15.82:0/2296894536 shutdown_connections 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 --2- 172.21.15.82:0/2296894536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810a260 0x7f337810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 --2- 172.21.15.82:0/2296894536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3378068510 0x7f33780fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 --2- 172.21.15.82:0/2296894536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33780ff390 0x7f33780ff800 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 -- 172.21.15.82:0/2296894536 >> 172.21.15.82:0/2296894536 conn(0x7f33780753a0 msgr2=0x7f33780757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 -- 172.21.15.82:0/2296894536 shutdown_connections 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.176+0000 7f337ebe3700 1 -- 172.21.15.82:0/2296894536 wait complete. 2023-07-14T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 Processor -- start 2023-07-14T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 -- start start 2023-07-14T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 0x7f3378100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33780ff390 0x7f3378100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810a260 0x7f33781a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a4490 con 0x7f3378068510 2023-07-14T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a45d0 con 0x7f337810a260 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337ebe3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a4740 con 0x7f33780ff390 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337c97f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 0x7f3378100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337c97f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 0x7f3378100210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43196/0 (socket says 172.21.15.82:43196) 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337c97f700 1 -- 172.21.15.82:0/1527691214 learned_addr learned my addr 172.21.15.82:0/1527691214 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.177+0000 7f337d180700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810a260 0x7f33781a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f3377fff700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33780ff390 0x7f3378100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337c97f700 1 -- 172.21.15.82:0/1527691214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33780ff390 msgr2=0x7f3378100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337c97f700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33780ff390 0x7f3378100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337c97f700 1 -- 172.21.15.82:0/1527691214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810a260 msgr2=0x7f33781a3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337c97f700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810a260 0x7f33781a3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337c97f700 1 -- 172.21.15.82:0/1527691214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33640095f0 con 0x7f3378068510 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f3377fff700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33780ff390 0x7f3378100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337c97f700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 0x7f3378100210 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f336c00d950 tx=0x7f336c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f3375ffb700 1 -- 172.21.15.82:0/1527691214 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c009940 con 0x7f3378068510 2023-07-14T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33781a4a20 con 0x7f3378068510 2023-07-14T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f3375ffb700 1 -- 172.21.15.82:0/1527691214 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336c010460 con 0x7f3378068510 2023-07-14T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f3375ffb700 1 -- 172.21.15.82:0/1527691214 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c00f620 con 0x7f3378068510 2023-07-14T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.178+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33781a4f80 con 0x7f3378068510 2023-07-14T19:29:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.180+0000 7f3375ffb700 1 -- 172.21.15.82:0/1527691214 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f336c0108d0 con 0x7f3378068510 2023-07-14T19:29:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.181+0000 7f3375ffb700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f33680a1460 0x7f33680a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.181+0000 7f3375ffb700 1 -- 172.21.15.82:0/1527691214 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f336c0ccd10 con 0x7f3378068510 2023-07-14T19:29:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.181+0000 7f3377fff700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f33680a1460 0x7f33680a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.181+0000 7f3377fff700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f33680a1460 0x7f33680a3910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f336400b5c0 tx=0x7f3364005fb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 Processor -- start 2023-07-14T19:29:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 -- start start 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d2814550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d280e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d280ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d2713c50 con 0x5649d266a6b0 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d27669e0 con 0x5649d2814a90 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d275fe70 con 0x5649d2814f00 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f3377fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d280e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f3377fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d280e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:39554/0 (socket says 172.21.15.82:39554) 2023-07-14T19:29:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f3377fff700 1 -- 172.21.15.82:0/19690133 learned_addr learned my addr 172.21.15.82:0/19690133 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.182+0000 7f337d180700 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d280ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f337d180700 1 -- 172.21.15.82:0/19690133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 msgr2=0x5649d280e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f337d180700 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d280e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f337d180700 1 -- 172.21.15.82:0/19690133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 msgr2=0x5649d2814550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f337d180700 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d2814550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f337d180700 1 -- 172.21.15.82:0/19690133 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d258dd80 con 0x5649d2814f00 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f337d180700 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d280ef80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3370003e80 tx=0x7f3370009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52985c0bd7585df server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f3362ffd700 1 -- 172.21.15.82:0/19690133 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370019070 con 0x5649d2814f00 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f3362ffd700 1 -- 172.21.15.82:0/19690133 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f337000f040 con 0x5649d2814f00 2023-07-14T19:29:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.183+0000 7f3362ffd700 1 -- 172.21.15.82:0/19690133 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370007960 con 0x5649d2814f00 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 -- 172.21.15.82:0/19690133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 msgr2=0x5649d280ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d280ef80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3370003e80 tx=0x7f3370009ba0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 -- 172.21.15.82:0/19690133 shutdown_connections 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d280ef80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d2814550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/19690133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d280e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 -- 172.21.15.82:0/19690133 >> 172.21.15.82:0/19690133 conn(0x5649d28043d0 msgr2=0x5649d2809530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 -- 172.21.15.82:0/19690133 shutdown_connections 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 -- 172.21.15.82:0/19690133 wait complete. 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 -- start start 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.184+0000 7f338ad7bb80 1 Processor -- start 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d266a6b0 0x5649d281ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d2818030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 0x5649d2818770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d26ffa60 con 0x5649d2814f00 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d26ff760 con 0x5649d266a6b0 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d25e38c0 con 0x5649d2814a90 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f338ad7bb80 10 client.0 _ll_register_callbacks cb 0x5649d26fd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 0x5649d2818770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 0x5649d2818770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43214/0 (socket says 172.21.15.82:43214) 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 -- 172.21.15.82:0/1671423928 learned_addr learned my addr 172.21.15.82:0/1671423928 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f3377fff700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d2818030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337c97f700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d266a6b0 0x5649d281ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 -- 172.21.15.82:0/1671423928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 msgr2=0x5649d2818030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d2818030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 -- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d266a6b0 msgr2=0x5649d281ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d266a6b0 0x5649d281ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.185+0000 7f337d180700 1 -- 172.21.15.82:0/1671423928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3370003ae0 con 0x5649d2814f00 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f337d180700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 0x5649d2818770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3370004780 tx=0x7f3370004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c275580241f851e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f3357fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f3357fff700 1 -- 172.21.15.82:0/1671423928 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370019040 con 0x5649d2814f00 2023-07-14T19:29:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f3357fff700 1 -- 172.21.15.82:0/1671423928 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f3370007a10 con 0x5649d2814f00 2023-07-14T19:29:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d26b9490 con 0x5649d2814f00 2023-07-14T19:29:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f3357fff700 1 -- 172.21.15.82:0/1671423928 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3370007bf0 con 0x5649d2814f00 2023-07-14T19:29:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 10 client.15648 fetch_fsmap learned FSMap version 25 2023-07-14T19:29:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5649d2576c00 con 0x5649d2814f00 2023-07-14T19:29:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f3357fff700 1 -- 172.21.15.82:0/1671423928 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649d26b9490 con 0x5649d2814f00 2023-07-14T19:29:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f3357fff700 1 -- 172.21.15.82:0/1671423928 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f337001f450 con 0x5649d2814f00 2023-07-14T19:29:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T19:29:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 20 client.15648 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d2828250 0x5649d282a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 4 client.15648 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.186+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649d27ddd40 con 0x5649d2828250 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.187+0000 7f337c97f700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d2828250 0x5649d282a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.187+0000 7f3357fff700 10 client.15648 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.187+0000 7f337c97f700 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d2828250 0x5649d282a660 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30a367e9ad6c0d6 server_cookie=ac835c98165d4a15 in_seq=0 out_seq=0 2023-07-14T19:29:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f3357fff700 1 -- 172.21.15.82:0/1671423928 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5649d27ddd40 con 0x5649d2828250 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f3357fff700 10 client.15648 handle_command_reply: tid=0 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 client.15648 shutdown 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 msgr2=0x5649d2818770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 0x5649d2818770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3370004780 tx=0x7f3370004860 comp rx=0 tx=0).stop 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 shutdown_connections 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d2814f00 0x5649d2818770 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d266a6b0 0x5649d281ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d2828250 0x5649d282a660 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f3357fff700 0 client.15648 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/1671423928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d2818030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 >> 172.21.15.82:0/1671423928 conn(0x5649d28043d0 msgr2=0x5649d2808c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 shutdown_connections 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 1 -- 172.21.15.82:0/1671423928 wait complete. 2023-07-14T19:29:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.188+0000 7f338ad7bb80 20 client.15648 trim_cache size 0 max 16384 2023-07-14T19:29:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.202+0000 7f338ad7bb80 1 Processor -- start 2023-07-14T19:29:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 -- start start 2023-07-14T19:29:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d2815a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d28093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d280fa10 con 0x5649d266a6b0 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d282f450 con 0x5649d2814a90 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d2818dd0 con 0x5649d2814f00 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43218/0 (socket says 172.21.15.82:43218) 2023-07-14T19:29:02.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 -- 172.21.15.82:0/2281601548 learned_addr learned my addr 172.21.15.82:0/2281601548 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f3377fff700 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d2815a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337d180700 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d28093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 -- 172.21.15.82:0/2281601548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 msgr2=0x5649d28093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d28093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 -- 172.21.15.82:0/2281601548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 msgr2=0x5649d2815a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d2815a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 -- 172.21.15.82:0/2281601548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d25b41f0 con 0x5649d266a6b0 2023-07-14T19:29:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337d180700 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d28093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.203+0000 7f337c97f700 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f3e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f336c096120 tx=0x7f336c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e071ece92d397c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f33637fe700 1 -- 172.21.15.82:0/2281601548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c09dc50 con 0x5649d266a6b0 2023-07-14T19:29:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f33637fe700 1 -- 172.21.15.82:0/2281601548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f336c0a2070 con 0x5649d266a6b0 2023-07-14T19:29:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f33637fe700 1 -- 172.21.15.82:0/2281601548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c099110 con 0x5649d266a6b0 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 -- 172.21.15.82:0/2281601548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 msgr2=0x5649d281f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f3e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f336c096120 tx=0x7f336c00bf20 comp rx=0 tx=0).stop 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 -- 172.21.15.82:0/2281601548 shutdown_connections 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814f00 0x5649d28093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f3e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/2281601548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814a90 0x5649d2815a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 -- 172.21.15.82:0/2281601548 >> 172.21.15.82:0/2281601548 conn(0x5649d28043d0 msgr2=0x5649d2815320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 -- 172.21.15.82:0/2281601548 shutdown_connections 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.204+0000 7f338ad7bb80 1 -- 172.21.15.82:0/2281601548 wait complete. 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 -- start start 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 Processor -- start 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d281b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814f00 0x5649d281be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d281c410 con 0x5649d266a6b0 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d281c580 con 0x5649d2814f00 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d281c740 con 0x5649d2814a90 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.205+0000 7f338ad7bb80 10 client.0 _ll_register_callbacks cb 0x7f3364005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43224/0 (socket says 172.21.15.82:43224) 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 -- 172.21.15.82:0/469766478 learned_addr learned my addr 172.21.15.82:0/469766478 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:02.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f3377fff700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d281b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337d180700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814f00 0x5649d281be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 -- 172.21.15.82:0/469766478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 msgr2=0x5649d281b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d281b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 -- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814f00 msgr2=0x5649d281be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814f00 0x5649d281be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 -- 172.21.15.82:0/469766478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33781a4f80 con 0x5649d266a6b0 2023-07-14T19:29:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f337c97f700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f336c0ca480 tx=0x7f336c099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4213935b207e8d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f3357fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f3357fff700 1 -- 172.21.15.82:0/469766478 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c099310 con 0x5649d266a6b0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f3357fff700 1 -- 172.21.15.82:0/469766478 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f336c098080 con 0x5649d266a6b0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f337000f040 con 0x5649d266a6b0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f3357fff700 1 -- 172.21.15.82:0/469766478 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f336c07c070 con 0x5649d266a6b0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f338ad7bb80 10 client.15654 fetch_fsmap learned FSMap version 25 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.206+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5649d258c250 con 0x5649d266a6b0 2023-07-14T19:29:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f3357fff700 1 -- 172.21.15.82:0/469766478 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f337000f040 con 0x5649d266a6b0 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f3357fff700 1 -- 172.21.15.82:0/469766478 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f336c0ca650 con 0x5649d266a6b0 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f338ad7bb80 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f338ad7bb80 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f338ad7bb80 20 client.15654 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d287cf90 0x5649d287f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f338ad7bb80 4 client.15654 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649d2758e50 con 0x5649d287cf90 2023-07-14T19:29:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f3377fff700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d287cf90 0x5649d287f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f3357fff700 10 client.15654 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:02.207+0000 7f3377fff700 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d287cf90 0x5649d287f3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7de6d167821dc64 server_cookie=f370626de0de89a4 in_seq=0 out_seq=0 2023-07-14T19:29:02.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:02 smithi161 ceph-mon[129184]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2023-07-14T19:29:02.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:02 smithi161 ceph-mon[129184]: mgrmap e23: x(active, since 28s), standbys: y, z 2023-07-14T19:29:02.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:02 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:02 smithi133 ceph-mon[128894]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2023-07-14T19:29:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:02 smithi133 ceph-mon[128894]: mgrmap e23: x(active, since 28s), standbys: y, z 2023-07-14T19:29:03.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:02 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:02 smithi082 ceph-mon[127461]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2023-07-14T19:29:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:02 smithi082 ceph-mon[127461]: mgrmap e23: x(active, since 28s), standbys: y, z 2023-07-14T19:29:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:02 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:04.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:04 smithi161 ceph-mon[129184]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T19:29:05.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:04 smithi133 ceph-mon[128894]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T19:29:05.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:04 smithi082 ceph-mon[127461]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-07-14T19:29:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f3357fff700 1 -- 172.21.15.82:0/469766478 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649d2758e50 con 0x5649d287cf90 2023-07-14T19:29:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f3357fff700 10 client.15654 handle_command_reply: tid=0 2023-07-14T19:29:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 client.15654 shutdown 2023-07-14T19:29:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 msgr2=0x5649d281f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f0a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f336c0ca480 tx=0x7f336c099b00 comp rx=0 tx=0).stop 2023-07-14T19:29:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 shutdown_connections 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5649d2814f00 0x5649d281be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649d266a6b0 0x5649d281f0a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5649d287cf90 0x5649d287f3a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 --2- 172.21.15.82:0/469766478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649d2814a90 0x5649d281b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f3357fff700 0 client.15654 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 >> 172.21.15.82:0/469766478 conn(0x5649d28043d0 msgr2=0x5649d28130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 shutdown_connections 2023-07-14T19:29:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 1 -- 172.21.15.82:0/469766478 wait complete. 2023-07-14T19:29:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.907+0000 7f338ad7bb80 20 client.15654 trim_cache size 0 max 16384 2023-07-14T19:29:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:29:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:29:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:29:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:29:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f33680a1460 msgr2=0x7f33680a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f33680a1460 0x7f33680a3910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f336400b5c0 tx=0x7f3364005fb0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 msgr2=0x7f3378100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 0x7f3378100210 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f336c00d950 tx=0x7f336c00dc60 comp rx=0 tx=0).stop 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 shutdown_connections 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f33680a1460 0x7f33680a3910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810a260 0x7f33781a3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3378068510 0x7f3378100210 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 --2- 172.21.15.82:0/1527691214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f33780ff390 0x7f3378100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 >> 172.21.15.82:0/1527691214 conn(0x7f33780753a0 msgr2=0x7f3378102170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 shutdown_connections 2023-07-14T19:29:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:05.908+0000 7f337ebe3700 1 -- 172.21.15.82:0/1527691214 wait complete. 2023-07-14T19:29:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:05 smithi133 ceph-mon[128894]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T19:29:06.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:05 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:active 2023-07-14T19:29:06.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:05 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:06.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:29:05.735+0000 7f9a2dc83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T19:29:06.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:05 smithi082 ceph-mon[127461]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T19:29:06.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:05 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:active 2023-07-14T19:29:06.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:05 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:05 smithi161 ceph-mon[129184]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-14T19:29:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:05 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] up:active 2023-07-14T19:29:06.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:05 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:06.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-mon[129184]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2023-07-14T19:29:06.992 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-mon[129184]: scrub summary: idle 2023-07-14T19:29:06.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-mon[129184]: scrub summary: idle 2023-07-14T19:29:06.993 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: *** Caught signal (Segmentation fault) ** 2023-07-14T19:29:06.993 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: in thread 7f774ae6a700 thread_name:md_log_replay 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f775ba18cf0] 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2: /usr/bin/ceph-mds(+0x4c68f2) [0x5623377d48f2] 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5623377d6b43] 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 4: (MDLog::_replay_thread()+0x66d) [0x562337842bad] 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5623374eeb61] 2023-07-14T19:29:06.994 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f775ba0e1ca] 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 7: clone() 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:06.723+0000 7f774ae6a700 -1 *** Caught signal (Segmentation fault) ** 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: in thread 7f774ae6a700 thread_name:md_log_replay 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f775ba18cf0] 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2: /usr/bin/ceph-mds(+0x4c68f2) [0x5623377d48f2] 2023-07-14T19:29:06.995 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5623377d6b43] 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 4: (MDLog::_replay_thread()+0x66d) [0x562337842bad] 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5623374eeb61] 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f775ba0e1ca] 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 7: clone() 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:06.996 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 0> 2023-07-14T19:29:06.723+0000 7f774ae6a700 -1 *** Caught signal (Segmentation fault) ** 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: in thread 7f774ae6a700 thread_name:md_log_replay 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f775ba18cf0] 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2: /usr/bin/ceph-mds(+0x4c68f2) [0x5623377d48f2] 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5623377d6b43] 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 4: (MDLog::_replay_thread()+0x66d) [0x562337842bad] 2023-07-14T19:29:06.997 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5623374eeb61] 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f775ba0e1ca] 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 7: clone() 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 0> 2023-07-14T19:29:06.723+0000 7f774ae6a700 -1 *** Caught signal (Segmentation fault) ** 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: in thread 7f774ae6a700 thread_name:md_log_replay 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:06.998 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: ceph version 18.0.0-4940-g81c9387d (81c9387d2e7576da065747ec9c16d1c7d69016ec) reef (dev) 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f775ba18cf0] 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2: /usr/bin/ceph-mds(+0x4c68f2) [0x5623377d48f2] 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5623377d6b43] 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 4: (MDLog::_replay_thread()+0x66d) [0x562337842bad] 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5623374eeb61] 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f775ba0e1ca] 2023-07-14T19:29:06.999 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 7: clone() 2023-07-14T19:29:07.000 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-14T19:29:07.000 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[154961]: 2023-07-14T19:29:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:06 smithi133 ceph-mon[128894]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2023-07-14T19:29:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:06 smithi133 ceph-mon[128894]: scrub summary: idle 2023-07-14T19:29:07.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:06 smithi133 ceph-mon[128894]: scrub summary: idle 2023-07-14T19:29:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:06 smithi082 ceph-mon[127461]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2023-07-14T19:29:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:06 smithi082 ceph-mon[127461]: scrub summary: idle 2023-07-14T19:29:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:06 smithi082 ceph-mon[127461]: scrub summary: idle 2023-07-14T19:29:07.300 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:07 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-07-14T19:29:07.491 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:07 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Failed with result 'exit-code'. 2023-07-14T19:29:09.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:08 smithi133 ceph-mon[128894]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-07-14T19:29:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:08 smithi082 ceph-mon[127461]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-07-14T19:29:09.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:08 smithi161 ceph-mon[129184]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-07-14T19:29:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:10 smithi133 ceph-mon[128894]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-07-14T19:29:11.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:10 smithi082 ceph-mon[127461]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-07-14T19:29:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:10 smithi161 ceph-mon[129184]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-07-14T19:29:13.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:12 smithi133 ceph-mon[128894]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2023-07-14T19:29:13.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:12 smithi082 ceph-mon[127461]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2023-07-14T19:29:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:12 smithi161 ceph-mon[129184]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2023-07-14T19:29:15.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:14 smithi133 ceph-mon[128894]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2023-07-14T19:29:15.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:14 smithi082 ceph-mon[127461]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2023-07-14T19:29:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:14 smithi161 ceph-mon[129184]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2023-07-14T19:29:17.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:16 smithi133 ceph-mon[128894]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-14T19:29:17.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:16 smithi082 ceph-mon[127461]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-14T19:29:17.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:16 smithi161 ceph-mon[129184]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-07-14T19:29:17.773 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:17 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-07-14T19:29:17.774 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:17 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Scheduled restart job, restart counter is at 1. 2023-07-14T19:29:17.774 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:17 smithi161 systemd[1]: Stopped Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:29:17.774 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:17 smithi161 systemd[1]: Starting Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:29:18.065 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:17 smithi161 podman[158376]: 2023-07-14T19:29:18.491 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:18 smithi161 ceph-mds[158416]: -- 172.21.15.161:0/1520283850 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558838dc11e0 con 0x558838d96c00 2023-07-14T19:29:18.492 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[158392]: starting mds.c at 2023-07-14T19:29:18.492 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:18 smithi161 bash[158376]: 3f6bffcb9cf80f58210496cb22e9a065a593df70b9c120d091327c366634f77f 2023-07-14T19:29:18.492 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:29:18 smithi161 systemd[1]: Started Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2980452684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 msgr2=0x7f6b4c1037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2980452684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c1037c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6b400099c0 tx=0x7f6b40009cd0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2980452684 shutdown_connections 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2980452684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b4c103ea0 0x7f6b4c10ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2980452684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c0fee60 0x7f6b4c0ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2980452684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c1037c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2980452684 >> 172.21.15.82:0/2980452684 conn(0x7f6b4c075d00 msgr2=0x7f6b4c076100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2980452684 shutdown_connections 2023-07-14T19:29:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.076+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2980452684 wait complete. 2023-07-14T19:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 Processor -- start 2023-07-14T19:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 -- start start 2023-07-14T19:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c0fee60 0x7f6b4c10c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c10c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b4c103ea0 0x7f6b4c1a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1a4470 con 0x7f6b4c0ff800 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1a45b0 con 0x7f6b4c0fee60 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.077+0000 7f6b50eaf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1a4720 con 0x7f6b4c103ea0 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c10c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c10c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43240/0 (socket says 172.21.15.82:43240) 2023-07-14T19:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2152631260 learned_addr learned my addr 172.21.15.82:0/2152631260 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b4c103ea0 0x7f6b4c1a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c0fee60 0x7f6b4c10c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2152631260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b4c103ea0 msgr2=0x7f6b4c1a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b4c103ea0 0x7f6b4c1a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2152631260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c0fee60 msgr2=0x7f6b4c10c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c0fee60 0x7f6b4c10c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2152631260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400095f0 con 0x7f6b4c0ff800 2023-07-14T19:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c10c8e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009990 tx=0x7f6b40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b3b7fe700 1 -- 172.21.15.82:0/2152631260 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4001d070 con 0x7f6b4c0ff800 2023-07-14T19:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b3b7fe700 1 -- 172.21.15.82:0/2152631260 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b400054f0 con 0x7f6b4c0ff800 2023-07-14T19:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b3b7fe700 1 -- 172.21.15.82:0/2152631260 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b40003ad0 con 0x7f6b4c0ff800 2023-07-14T19:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.078+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c1a49a0 con 0x7f6b4c0ff800 2023-07-14T19:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.079+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c1a4e00 con 0x7f6b4c0ff800 2023-07-14T19:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.082+0000 7f6b3b7fe700 1 -- 172.21.15.82:0/2152631260 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f6b400047a0 con 0x7f6b4c0ff800 2023-07-14T19:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.082+0000 7f6b3b7fe700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f6b340a13e0 0x7f6b340a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.083+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f6b340a13e0 0x7f6b340a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.083+0000 7f6b3b7fe700 1 -- 172.21.15.82:0/2152631260 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f6b400cdd00 con 0x7f6b4c0ff800 2023-07-14T19:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.083+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f6b340a13e0 0x7f6b340a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005950 tx=0x7f6b3c0058e0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 Processor -- start 2023-07-14T19:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 -- start start 2023-07-14T19:29:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a975393550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 0x55a97538d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97538df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a975292c50 con 0x55a975393a90 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9752e59e0 con 0x55a975393f00 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.084+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9752dee70 con 0x55a9751e96b0 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 0x55a97538d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 0x55a97538d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43248/0 (socket says 172.21.15.82:43248) 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2265401342 learned_addr learned my addr 172.21.15.82:0/2265401342 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a975393550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97538df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2265401342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 msgr2=0x55a975393550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a975393550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2265401342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 msgr2=0x55a97538df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97538df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 -- 172.21.15.82:0/2265401342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97510cd80 con 0x55a975393a90 2023-07-14T19:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97538df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.085+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 0x55a97538d9e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6b40012040 tx=0x7f6b40005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a6438faea55d84 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.086+0000 7f6b38ff9700 1 -- 172.21.15.82:0/2265401342 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4009ecd0 con 0x55a975393a90 2023-07-14T19:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.086+0000 7f6b38ff9700 1 -- 172.21.15.82:0/2265401342 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b400a3060 con 0x55a975393a90 2023-07-14T19:29:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.086+0000 7f6b38ff9700 1 -- 172.21.15.82:0/2265401342 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400cb640 con 0x55a975393a90 2023-07-14T19:29:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 -- 172.21.15.82:0/2265401342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 msgr2=0x55a97538d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 0x55a97538d9e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6b40012040 tx=0x7f6b40005fb0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 -- 172.21.15.82:0/2265401342 shutdown_connections 2023-07-14T19:29:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97538df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a975393550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/2265401342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393a90 0x55a97538d9e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 -- 172.21.15.82:0/2265401342 >> 172.21.15.82:0/2265401342 conn(0x55a9753833d0 msgr2=0x55a975388530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 -- 172.21.15.82:0/2265401342 shutdown_connections 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 -- 172.21.15.82:0/2265401342 wait complete. 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 -- start start 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.087+0000 7f6b5d047b80 1 Processor -- start 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a97539caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393a90 0x55a97539cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 0x55a975396da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97527ea60 con 0x55a975393f00 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97527e760 con 0x55a975393a90 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9751628c0 con 0x55a9751e96b0 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b5d047b80 10 client.0 _ll_register_callbacks cb 0x55a97527c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 0x55a975396da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 0x55a975396da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43256/0 (socket says 172.21.15.82:43256) 2023-07-14T19:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 -- 172.21.15.82:0/3460266419 learned_addr learned my addr 172.21.15.82:0/3460266419 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a97539caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393a90 0x55a97539cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 -- 172.21.15.82:0/3460266419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 msgr2=0x55a97539caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a97539caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 -- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393a90 msgr2=0x55a97539cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393a90 0x55a97539cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 -- 172.21.15.82:0/3460266419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c1a4e00 con 0x55a975393f00 2023-07-14T19:29:19.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 0x55a975396da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009fc0 tx=0x7f6b440043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90d6f7f5df44af5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.088+0000 7f6b31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b31ffb700 1 -- 172.21.15.82:0/3460266419 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440079d0 con 0x55a975393f00 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b31ffb700 1 -- 172.21.15.82:0/3460266419 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f6b44007b30 con 0x55a975393f00 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a975238490 con 0x55a975393f00 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b31ffb700 1 -- 172.21.15.82:0/3460266419 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b44007cf0 con 0x55a975393f00 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 10 client.15672 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a9750f5c00 con 0x55a975393f00 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b31ffb700 1 -- 172.21.15.82:0/3460266419 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a975238490 con 0x55a975393f00 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b31ffb700 1 -- 172.21.15.82:0/3460266419 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b44014040 con 0x55a975393f00 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 20 client.15672 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a9753a70d0 0x55a9753a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 4 client.15672 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:19.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.089+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a97535cd40 con 0x55a9753a70d0 2023-07-14T19:29:19.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.090+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a9753a70d0 0x55a9753a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.091+0000 7f6b31ffb700 10 client.15672 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.091+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a9753a70d0 0x55a9753a94e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500e719e08520a63 server_cookie=4eb2b27c30ff69d in_seq=0 out_seq=0 2023-07-14T19:29:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.091+0000 7f6b31ffb700 1 -- 172.21.15.82:0/3460266419 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a97535cd40 con 0x55a9753a70d0 2023-07-14T19:29:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.091+0000 7f6b31ffb700 10 client.15672 handle_command_reply: tid=0 2023-07-14T19:29:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.091+0000 7f6b5d047b80 1 client.15672 shutdown 2023-07-14T19:29:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 msgr2=0x55a975396da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 0x55a975396da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009fc0 tx=0x7f6b440043d0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 shutdown_connections 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a975393f00 0x55a975396da0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a9753a70d0 0x55a9753a94e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9751e96b0 0x55a97539caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/3460266419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393a90 0x55a97539cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 >> 172.21.15.82:0/3460266419 conn(0x55a9753833d0 msgr2=0x55a975387c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.092+0000 7f6b31ffb700 0 client.15672 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.093+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 shutdown_connections 2023-07-14T19:29:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.093+0000 7f6b5d047b80 1 -- 172.21.15.82:0/3460266419 wait complete. 2023-07-14T19:29:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.093+0000 7f6b5d047b80 20 client.15672 trim_cache size 0 max 16384 2023-07-14T19:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.120+0000 7f6b5d047b80 1 Processor -- start 2023-07-14T19:29:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.120+0000 7f6b5d047b80 1 -- start start 2023-07-14T19:29:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a97538d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a97538df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a975396580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9753805c0 con 0x55a9751e96b0 2023-07-14T19:29:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a975395320 con 0x55a975393f00 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97538a3c0 con 0x55a975393a90 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a97538d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a97538d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43260/0 (socket says 172.21.15.82:43260) 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 -- 172.21.15.82:0/305657533 learned_addr learned my addr 172.21.15.82:0/305657533 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a97538df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a975396580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 -- 172.21.15.82:0/305657533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 msgr2=0x55a97538df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a97538df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 -- 172.21.15.82:0/305657533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 msgr2=0x55a975396580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a975396580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.121+0000 7f6b4a59c700 1 -- 172.21.15.82:0/305657533 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9751331f0 con 0x55a9751e96b0 2023-07-14T19:29:19.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.122+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a97538d9e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00b570 tx=0x7f6b3c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9352775948ebaf08 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.122+0000 7f6b397fa700 1 -- 172.21.15.82:0/305657533 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00fac0 con 0x55a9751e96b0 2023-07-14T19:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.122+0000 7f6b397fa700 1 -- 172.21.15.82:0/305657533 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b3c028c80 con 0x55a9751e96b0 2023-07-14T19:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.122+0000 7f6b397fa700 1 -- 172.21.15.82:0/305657533 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c011770 con 0x55a9751e96b0 2023-07-14T19:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 -- 172.21.15.82:0/305657533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 msgr2=0x55a97538d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a97538d9e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00b570 tx=0x7f6b3c015000 comp rx=0 tx=0).stop 2023-07-14T19:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 -- 172.21.15.82:0/305657533 shutdown_connections 2023-07-14T19:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a975396580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a97538d9e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/305657533 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a97538df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.123+0000 7f6b5d047b80 1 -- 172.21.15.82:0/305657533 >> 172.21.15.82:0/305657533 conn(0x55a9753833d0 msgr2=0x55a9753911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 -- 172.21.15.82:0/305657533 shutdown_connections 2023-07-14T19:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 -- 172.21.15.82:0/305657533 wait complete. 2023-07-14T19:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 -- start start 2023-07-14T19:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 Processor -- start 2023-07-14T19:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a9753f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a9753f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97539a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9753f6a80 con 0x55a9751e96b0 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97539acd0 con 0x55a975393f00 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97539ae90 con 0x55a975393a90 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.127+0000 7f6b5d047b80 10 client.0 _ll_register_callbacks cb 0x7f6b440098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a9753f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a9753f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:52530/0 (socket says 172.21.15.82:52530) 2023-07-14T19:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 -- 172.21.15.82:0/4135575456 learned_addr learned my addr 172.21.15.82:0/4135575456 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97539a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 -- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 msgr2=0x55a97539a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97539a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 -- 172.21.15.82:0/4135575456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 msgr2=0x55a9753f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a9753f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 -- 172.21.15.82:0/4135575456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c00fe80 con 0x55a975393a90 2023-07-14T19:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b4ad9d700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97539a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b49d9b700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a9753f6250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b40097710 tx=0x7f6b40000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4b77d20c21d9d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.128+0000 7f6b31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b31ffb700 1 -- 172.21.15.82:0/4135575456 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400cb2a0 con 0x55a975393a90 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b400a3060 con 0x55a975393a90 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b31ffb700 1 -- 172.21.15.82:0/4135575456 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f6b4009ecd0 con 0x55a975393a90 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b31ffb700 1 -- 172.21.15.82:0/4135575456 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b400b7030 con 0x55a975393a90 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 10 client.25264 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a97510b250 con 0x55a975393a90 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b31ffb700 1 -- 172.21.15.82:0/4135575456 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b400c9080 con 0x55a975393a90 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b31ffb700 1 -- 172.21.15.82:0/4135575456 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b400b32b0 con 0x55a975393a90 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 10 client.25264 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 10 client.25264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 20 client.25264 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a97540c670 0x55a97540ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 4 client.25264 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.129+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a9752d7e50 con 0x55a97540c670 2023-07-14T19:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.130+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a97540c670 0x55a97540ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.130+0000 7f6b31ffb700 10 client.25264 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.130+0000 7f6b4a59c700 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a97540c670 0x55a97540ea80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be5ea31574db723 server_cookie=c08c5a4afac98f07 in_seq=0 out_seq=0 2023-07-14T19:29:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.135+0000 7f6b31ffb700 1 -- 172.21.15.82:0/4135575456 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a9752d7e50 con 0x55a97540c670 2023-07-14T19:29:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.135+0000 7f6b31ffb700 10 client.25264 handle_command_reply: tid=0 2023-07-14T19:29:19.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.135+0000 7f6b5d047b80 1 client.25264 shutdown 2023-07-14T19:29:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 msgr2=0x55a9753f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a9753f6250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b40097710 tx=0x7f6b40000bc0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 shutdown_connections 2023-07-14T19:29:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a975393f00 0x55a97539a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9751e96b0 0x55a9753f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a97540c670 0x55a97540ea80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 --2- 172.21.15.82:0/4135575456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a975393a90 0x55a9753f6250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b31ffb700 0 client.25264 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.136+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 >> 172.21.15.82:0/4135575456 conn(0x55a9753833d0 msgr2=0x55a975391e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 shutdown_connections 2023-07-14T19:29:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b5d047b80 1 -- 172.21.15.82:0/4135575456 wait complete. 2023-07-14T19:29:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b5d047b80 20 client.25264 trim_cache size 0 max 16384 2023-07-14T19:29:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:29:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:29:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:29:19.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:29:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f6b340a13e0 msgr2=0x7f6b340a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f6b340a13e0 0x7f6b340a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c005950 tx=0x7f6b3c0058e0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 msgr2=0x7f6b4c10c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c10c8e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009990 tx=0x7f6b40004730 comp rx=0 tx=0).stop 2023-07-14T19:29:19.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 shutdown_connections 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f6b340a13e0 0x7f6b340a3890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b4c103ea0 0x7f6b4c1a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b4c0fee60 0x7f6b4c10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 --2- 172.21.15.82:0/2152631260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b4c0ff800 0x7f6b4c10c8e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.137+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 >> 172.21.15.82:0/2152631260 conn(0x7f6b4c075d00 msgr2=0x7f6b4c0fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.138+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 shutdown_connections 2023-07-14T19:29:19.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:19.138+0000 7f6b50eaf700 1 -- 172.21.15.82:0/2152631260 wait complete. 2023-07-14T19:29:19.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:18 smithi133 ceph-mon[128894]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 341 B/s wr, 4 op/s 2023-07-14T19:29:19.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:18 smithi133 ceph-mon[128894]: daemon mds.c restarted 2023-07-14T19:29:19.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:18 smithi082 ceph-mon[127461]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 341 B/s wr, 4 op/s 2023-07-14T19:29:19.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:18 smithi082 ceph-mon[127461]: daemon mds.c restarted 2023-07-14T19:29:19.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:18 smithi161 ceph-mon[129184]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 341 B/s wr, 4 op/s 2023-07-14T19:29:19.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:18 smithi161 ceph-mon[129184]: daemon mds.c restarted 2023-07-14T19:29:20.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:19 smithi133 ceph-mon[128894]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T19:29:20.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:19 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6836/4285297228,v1:172.21.15.161:6837/4285297228] up:boot 2023-07-14T19:29:20.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:19 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:29:20.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:19 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:29:20.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:19 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:20.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:19 smithi082 ceph-mon[127461]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T19:29:20.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:19 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6836/4285297228,v1:172.21.15.161:6837/4285297228] up:boot 2023-07-14T19:29:20.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:19 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:29:20.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:19 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:29:20.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:19 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:19 smithi161 ceph-mon[129184]: osdmap e85: 12 total, 12 up, 12 in 2023-07-14T19:29:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:19 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6836/4285297228,v1:172.21.15.161:6837/4285297228] up:boot 2023-07-14T19:29:20.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:19 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:29:20.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:19 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:29:20.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:19 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:21.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:20 smithi133 ceph-mon[128894]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 409 B/s wr, 5 op/s 2023-07-14T19:29:21.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:20 smithi082 ceph-mon[127461]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 409 B/s wr, 5 op/s 2023-07-14T19:29:21.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:20 smithi161 ceph-mon[129184]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 409 B/s wr, 5 op/s 2023-07-14T19:29:23.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:22 smithi133 ceph-mon[128894]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 5 op/s 2023-07-14T19:29:23.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:22 smithi082 ceph-mon[127461]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 5 op/s 2023-07-14T19:29:23.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:22 smithi161 ceph-mon[129184]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 5 op/s 2023-07-14T19:29:25.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:24 smithi133 ceph-mon[128894]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 5 op/s 2023-07-14T19:29:25.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:24 smithi082 ceph-mon[127461]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 5 op/s 2023-07-14T19:29:25.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:24 smithi161 ceph-mon[129184]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 5 op/s 2023-07-14T19:29:27.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:26 smithi133 ceph-mon[128894]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-07-14T19:29:27.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:26 smithi082 ceph-mon[127461]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-07-14T19:29:27.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:26 smithi161 ceph-mon[129184]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-07-14T19:29:29.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-14T19:29:29.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:28 smithi133 ceph-mon[128894]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-07-14T19:29:29.195 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:28 smithi082 ceph-mon[127461]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-07-14T19:29:29.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:28 smithi161 ceph-mon[129184]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-07-14T19:29:29.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 -- 172.21.15.82:0/1587661016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fecc0 msgr2=0x7f1b9410caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 --2- 172.21.15.82:0/1587661016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fecc0 0x7f1b9410caf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b0a0 tx=0x7f1b8c00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 -- 172.21.15.82:0/1587661016 shutdown_connections 2023-07-14T19:29:29.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 --2- 172.21.15.82:0/1587661016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fecc0 0x7f1b9410caf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 --2- 172.21.15.82:0/1587661016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b94105d30 0x7f1b94106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 --2- 172.21.15.82:0/1587661016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b940fe330 0x7f1b940fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 -- 172.21.15.82:0/1587661016 >> 172.21.15.82:0/1587661016 conn(0x7f1b94076330 msgr2=0x7f1b94076730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 -- 172.21.15.82:0/1587661016 shutdown_connections 2023-07-14T19:29:29.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 -- 172.21.15.82:0/1587661016 wait complete. 2023-07-14T19:29:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.302+0000 7f1b9beef700 1 Processor -- start 2023-07-14T19:29:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 -- start start 2023-07-14T19:29:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 0x7f1b94195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b940fecc0 0x7f1b94196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94105d30 0x7f1b9419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.307 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9419af20 con 0x7f1b940fe330 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9419b090 con 0x7f1b94105d30 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9419b250 con 0x7f1b940fecc0 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 0x7f1b94195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 0x7f1b94195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43278/0 (socket says 172.21.15.82:43278) 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 -- 172.21.15.82:0/423256093 learned_addr learned my addr 172.21.15.82:0/423256093 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 -- 172.21.15.82:0/423256093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b940fecc0 msgr2=0x7f1b94196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9948a700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b940fecc0 0x7f1b94196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94105d30 0x7f1b9419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b940fecc0 0x7f1b94196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 -- 172.21.15.82:0/423256093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94105d30 msgr2=0x7f1b9419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94105d30 0x7f1b9419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 -- 172.21.15.82:0/423256093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c009d00 con 0x7f1b940fe330 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94105d30 0x7f1b9419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:29.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 0x7f1b94195f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c920 tx=0x7f1b9000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b8affd700 1 -- 172.21.15.82:0/423256093 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90007b00 con 0x7f1b940fe330 2023-07-14T19:29:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b8affd700 1 -- 172.21.15.82:0/423256093 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b90004cf0 con 0x7f1b940fe330 2023-07-14T19:29:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9419b530 con 0x7f1b940fe330 2023-07-14T19:29:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b8affd700 1 -- 172.21.15.82:0/423256093 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90005610 con 0x7f1b940fe330 2023-07-14T19:29:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.303+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b941a2d70 con 0x7f1b940fe330 2023-07-14T19:29:29.310 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.305+0000 7f1b8affd700 1 -- 172.21.15.82:0/423256093 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f1b90004750 con 0x7f1b940fe330 2023-07-14T19:29:29.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.306+0000 7f1b8affd700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b800a13e0 0x7f1b800a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.306+0000 7f1b9948a700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b800a13e0 0x7f1b800a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.306+0000 7f1b8affd700 1 -- 172.21.15.82:0/423256093 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1b900cceb0 con 0x7f1b940fe330 2023-07-14T19:29:29.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1b9948a700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b800a13e0 0x7f1b800a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b94197550 tx=0x7f1b84005cf0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 Processor -- start 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 -- start start 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f5991c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f59921080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59820cb0 con 0x563f5991c790 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f59873a40 con 0x563f5991be80 2023-07-14T19:29:29.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5986ced0 con 0x563f59920c30 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.307+0000 7f1b9948a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599206f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43288/0 (socket says 172.21.15.82:43288) 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 -- 172.21.15.82:0/234896475 learned_addr learned my addr 172.21.15.82:0/234896475 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f5991c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f59921080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 -- 172.21.15.82:0/234896475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 msgr2=0x563f59921080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f59921080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.317 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 -- 172.21.15.82:0/234896475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 msgr2=0x563f5991c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f5991c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 -- 172.21.15.82:0/234896475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f5969ade0 con 0x563f5991c790 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b9948a700 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599206f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400dbb0 tx=0x7f1b84010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255d9b14336ab968 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b7ffff700 1 -- 172.21.15.82:0/234896475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84004a30 con 0x563f5991c790 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b7ffff700 1 -- 172.21.15.82:0/234896475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b840054e0 con 0x563f5991c790 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.308+0000 7f1b7ffff700 1 -- 172.21.15.82:0/234896475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400fc10 con 0x563f5991c790 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 -- 172.21.15.82:0/234896475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 msgr2=0x563f599206f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.318 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599206f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400dbb0 tx=0x7f1b84010000 comp rx=0 tx=0).stop 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 -- 172.21.15.82:0/234896475 shutdown_connections 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f59921080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f5991c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/234896475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599206f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 -- 172.21.15.82:0/234896475 >> 172.21.15.82:0/234896475 conn(0x563f59911680 msgr2=0x563f59916ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 -- 172.21.15.82:0/234896475 shutdown_connections 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 -- 172.21.15.82:0/234896475 wait complete. 2023-07-14T19:29:29.319 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 -- start start 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.309+0000 7f1ba8087b80 1 Processor -- start 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991be80 0x563f59928410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991c790 0x563f59925620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 0x563f59925d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5980cac0 con 0x563f59920c30 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5980c7c0 con 0x563f5991c790 2023-07-14T19:29:29.320 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f596f0920 con 0x563f5991be80 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 10 client.0 _ll_register_callbacks cb 0x563f5980a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 0x563f59925d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 0x563f59925d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43296/0 (socket says 172.21.15.82:43296) 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 -- 172.21.15.82:0/4009358954 learned_addr learned my addr 172.21.15.82:0/4009358954 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9948a700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991c790 0x563f59925620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991be80 0x563f59928410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 -- 172.21.15.82:0/4009358954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991be80 msgr2=0x563f59928410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.321 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991be80 0x563f59928410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 -- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991c790 msgr2=0x563f59925620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991c790 0x563f59925620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 -- 172.21.15.82:0/4009358954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8400d860 con 0x563f59920c30 2023-07-14T19:29:29.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9948a700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991c790 0x563f59925620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:29.322 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 0x563f59925d10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b070 tx=0x7f1b8c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1ad4b3fa9c55c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/4009358954 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00e050 con 0x563f59920c30 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/4009358954 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1b8c0078f0 con 0x563f59920c30 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.310+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f597c64f0 con 0x563f59920c30 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/4009358954 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b8c005040 con 0x563f59920c30 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 10 client.15687 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f59683c60 con 0x563f59920c30 2023-07-14T19:29:29.323 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/4009358954 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f597c64f0 con 0x563f59920c30 2023-07-14T19:29:29.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/4009358954 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b8c00b0d0 con 0x563f59920c30 2023-07-14T19:29:29.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:29.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:29.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 20 client.15687 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:29.324 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f59935c40 0x563f59938050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 4 client.15687 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f598eada0 con 0x563f59935c40 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.311+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f59935c40 0x563f59938050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1b7d7fa700 10 client.15687 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f59935c40 0x563f59938050 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38afaf7849bf766 server_cookie=b3f807c6f1afda51 in_seq=0 out_seq=0 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/4009358954 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563f598eada0 con 0x563f59935c40 2023-07-14T19:29:29.325 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1b7d7fa700 10 client.15687 handle_command_reply: tid=0 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 client.15687 shutdown 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 msgr2=0x563f59925d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 0x563f59925d10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b070 tx=0x7f1b8c0095d0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 shutdown_connections 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f59920c30 0x563f59925d10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f59935c40 0x563f59938050 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.326 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991be80 0x563f59928410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1b7d7fa700 0 client.15687 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:29.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/4009358954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991c790 0x563f59925620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.312+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 >> 172.21.15.82:0/4009358954 conn(0x563f59911680 msgr2=0x563f59916a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.313+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 shutdown_connections 2023-07-14T19:29:29.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.313+0000 7f1ba8087b80 1 -- 172.21.15.82:0/4009358954 wait complete. 2023-07-14T19:29:29.327 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.313+0000 7f1ba8087b80 20 client.15687 trim_cache size 0 max 16384 2023-07-14T19:29:29.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 Processor -- start 2023-07-14T19:29:29.335 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 -- start start 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 0x563f5991e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991c790 0x563f5992cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f59920c30 0x563f59927f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f599260d0 con 0x563f5991be80 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f599332d0 con 0x563f59920c30 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5991cf40 con 0x563f5991c790 2023-07-14T19:29:29.336 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 0x563f5991e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 0x563f5991e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43300/0 (socket says 172.21.15.82:43300) 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 -- 172.21.15.82:0/1977945969 learned_addr learned my addr 172.21.15.82:0/1977945969 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f59920c30 0x563f59927f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 -- 172.21.15.82:0/1977945969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991c790 msgr2=0x563f5992cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991c790 0x563f5992cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 -- 172.21.15.82:0/1977945969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f59920c30 msgr2=0x563f59927f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f59920c30 0x563f59927f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.337 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 -- 172.21.15.82:0/1977945969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f596c1250 con 0x563f5991be80 2023-07-14T19:29:29.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.333+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 0x563f5991e520 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b90007fd0 tx=0x7f1b90099b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c094024de7f9a8ed server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1b88ff9700 1 -- 172.21.15.82:0/1977945969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90097f00 con 0x563f5991be80 2023-07-14T19:29:29.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1b88ff9700 1 -- 172.21.15.82:0/1977945969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b90098160 con 0x563f5991be80 2023-07-14T19:29:29.338 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1b88ff9700 1 -- 172.21.15.82:0/1977945969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90004b90 con 0x563f5991be80 2023-07-14T19:29:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1977945969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 msgr2=0x563f5991e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 0x563f5991e520 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b90007fd0 tx=0x7f1b90099b90 comp rx=0 tx=0).stop 2023-07-14T19:29:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1977945969 shutdown_connections 2023-07-14T19:29:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f59920c30 0x563f59927f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991be80 0x563f5991e520 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1977945969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f5991c790 0x563f5992cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1977945969 >> 172.21.15.82:0/1977945969 conn(0x563f59911680 msgr2=0x563f5991ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1977945969 shutdown_connections 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.334+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1977945969 wait complete. 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 -- start start 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 Processor -- start 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f59983280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f5997d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5997de50 con 0x563f5991c790 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5997dfc0 con 0x563f5991be80 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5997e180 con 0x563f59920c30 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1ba8087b80 10 client.0 _ll_register_callbacks cb 0x7f1b8c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1b9948a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1b9948a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599837c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43308/0 (socket says 172.21.15.82:43308) 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1b9948a700 1 -- 172.21.15.82:0/1839389261 learned_addr learned my addr 172.21.15.82:0/1839389261 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f59983280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.335+0000 7f1b9a48c700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f5997d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b9948a700 1 -- 172.21.15.82:0/1839389261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 msgr2=0x563f5997d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b9948a700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f5997d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b9948a700 1 -- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 msgr2=0x563f59983280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b9948a700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f59983280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b9948a700 1 -- 172.21.15.82:0/1839389261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b941a2d70 con 0x563f5991c790 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f59983280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b9948a700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599837c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400dbb0 tx=0x7f1b8400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f4e0ab89c2723c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/1839389261 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840056c0 con 0x563f5991c790 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/1839389261 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1b840046f0 con 0x563f5991c790 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b840054e0 con 0x563f5991c790 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/1839389261 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b840048b0 con 0x563f5991c790 2023-07-14T19:29:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1ba8087b80 10 client.15693 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f596992b0 con 0x563f5991c790 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.336+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/1839389261 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b8403e020 con 0x563f5991c790 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/1839389261 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b84030ba0 con 0x563f5991c790 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1ba8087b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1ba8087b80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1ba8087b80 20 client.15693 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f5998b0c0 0x563f5998d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1ba8087b80 4 client.15693 mds_command: new command op to 24524 tid=0[{"prefix": "scrub status"}] 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f59865eb0 con 0x563f5998b0c0 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f5998b0c0 0x563f5998d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1b7d7fa700 10 client.15693 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.337+0000 7f1b99c8b700 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f5998b0c0 0x563f5998d4d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb5c209c4d802a0 server_cookie=ec944486c1f0672f in_seq=0 out_seq=0 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b7d7fa700 1 -- 172.21.15.82:0/1839389261 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563f59865eb0 con 0x563f5998b0c0 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b7d7fa700 10 client.15693 handle_command_reply: tid=0 2023-07-14T19:29:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 client.15693 shutdown 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 msgr2=0x563f599837c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599837c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400dbb0 tx=0x7f1b8400f9b0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 shutdown_connections 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f5998b0c0 0x563f5998d4d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f59920c30 0x563f5997d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f5991be80 0x563f59983280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b7d7fa700 0 client.15693 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 --2- 172.21.15.82:0/1839389261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f5991c790 0x563f599837c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 >> 172.21.15.82:0/1839389261 conn(0x563f59911680 msgr2=0x563f5991fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 shutdown_connections 2023-07-14T19:29:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 1 -- 172.21.15.82:0/1839389261 wait complete. 2023-07-14T19:29:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1ba8087b80 20 client.15693 trim_cache size 0 max 16384 2023-07-14T19:29:29.354 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:29:29.354 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2023-07-14T19:29:29.354 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2023-07-14T19:29:29.354 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:29:29.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b800a13e0 msgr2=0x7f1b800a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.355 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b9beef700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b800a13e0 0x7f1b800a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b94197550 tx=0x7f1b84005cf0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 msgr2=0x7f1b94195f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.338+0000 7f1b9beef700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 0x7f1b94195f40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c920 tx=0x7f1b9000cce0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 shutdown_connections 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b800a13e0 0x7f1b800a3890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94105d30 0x7f1b9419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940fe330 0x7f1b94195f40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.356 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 --2- 172.21.15.82:0/423256093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b940fecc0 0x7f1b94196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 >> 172.21.15.82:0/423256093 conn(0x7f1b94076330 msgr2=0x7f1b940fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 shutdown_connections 2023-07-14T19:29:29.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.339+0000 7f1b9beef700 1 -- 172.21.15.82:0/423256093 wait complete. 2023-07-14T19:29:29.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-14T19:29:29.357 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:29:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 -- 172.21.15.82:0/2562125410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 msgr2=0x7f01780fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 --2- 172.21.15.82:0/2562125410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f01780fef70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0168009990 tx=0x7f0168009ca0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 -- 172.21.15.82:0/2562125410 shutdown_connections 2023-07-14T19:29:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 --2- 172.21.15.82:0/2562125410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01780ff4b0 0x7f017810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 --2- 172.21.15.82:0/2562125410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0178104b10 0x7f0178104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 --2- 172.21.15.82:0/2562125410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f01780fef70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 -- 172.21.15.82:0/2562125410 >> 172.21.15.82:0/2562125410 conn(0x7f01780fa5c0 msgr2=0x7f01780fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 -- 172.21.15.82:0/2562125410 shutdown_connections 2023-07-14T19:29:29.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.516+0000 7f017f4fb700 1 -- 172.21.15.82:0/2562125410 wait complete. 2023-07-14T19:29:29.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 Processor -- start 2023-07-14T19:29:29.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 -- start start 2023-07-14T19:29:29.521 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f017819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01780ff4b0 0x7f017819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0178104b10 0x7f0178195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0178196140 con 0x7f01780feb00 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01781962b0 con 0x7f01780ff4b0 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017f4fb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0178196470 con 0x7f0178104b10 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f017819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.522 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f017819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43320/0 (socket says 172.21.15.82:43320) 2023-07-14T19:29:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 -- 172.21.15.82:0/1101108672 learned_addr learned my addr 172.21.15.82:0/1101108672 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017ca96700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01780ff4b0 0x7f017819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017da98700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0178104b10 0x7f0178195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 -- 172.21.15.82:0/1101108672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0178104b10 msgr2=0x7f0178195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0178104b10 0x7f0178195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 -- 172.21.15.82:0/1101108672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01780ff4b0 msgr2=0x7f017819bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01780ff4b0 0x7f017819bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.517+0000 7f017d297700 1 -- 172.21.15.82:0/1101108672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01680095f0 con 0x7f01780feb00 2023-07-14T19:29:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.518+0000 7f017d297700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f017819ba60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f017400d950 tx=0x7f017400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.518+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0174009940 con 0x7f01780feb00 2023-07-14T19:29:29.524 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.518+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0174010460 con 0x7f01780feb00 2023-07-14T19:29:29.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.518+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0178196750 con 0x7f01780feb00 2023-07-14T19:29:29.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.518+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400f620 con 0x7f01780feb00 2023-07-14T19:29:29.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.518+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01781a2a50 con 0x7f01780feb00 2023-07-14T19:29:29.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.519+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0178067430 con 0x7f01780feb00 2023-07-14T19:29:29.525 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.520+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f01740108d0 con 0x7f01780feb00 2023-07-14T19:29:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.520+0000 7f016e7fc700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f01640a1430 0x7f01640a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.521+0000 7f017ca96700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f01640a1430 0x7f01640a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.521+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f01740ccd30 con 0x7f01780feb00 2023-07-14T19:29:29.526 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.521+0000 7f017ca96700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f01640a1430 0x7f01640a38e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01781041f0 tx=0x7f0168005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.527 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.524+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0174095f80 con 0x7f01780feb00 2023-07-14T19:29:29.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.759+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0178196fe0 con 0x7f01780feb00 2023-07-14T19:29:29.761 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.760+0000 7f016e7fc700 1 -- 172.21.15.82:0/1101108672 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10835 (secure 0 0 0) 0x7f0174016090 con 0x7f01780feb00 2023-07-14T19:29:29.762 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:29:29.762 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:29:18.822943+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":37,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25249":{"gid":25249,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/4285297228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":4285297228},{"type":"v1","addr":"172.21.15.161:6837","nonce":4285297228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:29:29.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.762+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f01640a1430 msgr2=0x7f01640a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.762+0000 7f017f4fb700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f01640a1430 0x7f01640a38e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f01781041f0 tx=0x7f0168005fd0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.762+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 msgr2=0x7f017819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.762+0000 7f017f4fb700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f017819ba60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f017400d950 tx=0x7f017400dc60 comp rx=0 tx=0).stop 2023-07-14T19:29:29.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 shutdown_connections 2023-07-14T19:29:29.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f01640a1430 0x7f01640a38e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0178104b10 0x7f0178195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01780feb00 0x7f017819ba60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 --2- 172.21.15.82:0/1101108672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01780ff4b0 0x7f017819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 >> 172.21.15.82:0/1101108672 conn(0x7f01780fa5c0 msgr2=0x7f01780fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.765 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 shutdown_connections 2023-07-14T19:29:29.766 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.763+0000 7f017f4fb700 1 -- 172.21.15.82:0/1101108672 wait complete. 2023-07-14T19:29:29.766 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 28 2023-07-14T19:29:29.778 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-14T19:29:29.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 -- 172.21.15.82:0/2157745061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58104ef0 msgr2=0x7f1b5810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/2157745061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58104ef0 0x7f1b5810cac0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b0d0 tx=0x7f1b5000b3e0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 -- 172.21.15.82:0/2157745061 shutdown_connections 2023-07-14T19:29:29.905 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/2157745061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58104ef0 0x7f1b5810cac0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/2157745061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b580ffeb0 0x7f1b58100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/2157745061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b58100850 0x7f1b58104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 -- 172.21.15.82:0/2157745061 >> 172.21.15.82:0/2157745061 conn(0x7f1b580fb760 msgr2=0x7f1b580fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 -- 172.21.15.82:0/2157745061 shutdown_connections 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 -- 172.21.15.82:0/2157745061 wait complete. 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 Processor -- start 2023-07-14T19:29:29.906 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.902+0000 7f1b5f85b700 1 -- start start 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b580ffeb0 0x7f1b58195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 0x7f1b58196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b58104ef0 0x7f1b5819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5819add0 con 0x7f1b58100850 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5819af40 con 0x7f1b580ffeb0 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5819b100 con 0x7f1b58104ef0 2023-07-14T19:29:29.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 0x7f1b58196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 0x7f1b58196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43334/0 (socket says 172.21.15.82:43334) 2023-07-14T19:29:29.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/3168812710 learned_addr learned my addr 172.21.15.82:0/3168812710 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/3168812710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b58104ef0 msgr2=0x7f1b5819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b58104ef0 0x7f1b5819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.908 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/3168812710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b580ffeb0 msgr2=0x7f1b58195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b580ffeb0 0x7f1b58195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/3168812710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50009d00 con 0x7f1b58100850 2023-07-14T19:29:29.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 0x7f1b58196330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800d950 tx=0x7f1b4800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.909 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b4e7fc700 1 -- 172.21.15.82:0/3168812710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48009b40 con 0x7f1b58100850 2023-07-14T19:29:29.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b4e7fc700 1 -- 172.21.15.82:0/3168812710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4800de30 con 0x7f1b58100850 2023-07-14T19:29:29.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5819b3e0 con 0x7f1b58100850 2023-07-14T19:29:29.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b4e7fc700 1 -- 172.21.15.82:0/3168812710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800f620 con 0x7f1b58100850 2023-07-14T19:29:29.910 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.903+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b581a2c20 con 0x7f1b58100850 2023-07-14T19:29:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.906+0000 7f1b4e7fc700 1 -- 172.21.15.82:0/3168812710 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f1b48010460 con 0x7f1b58100850 2023-07-14T19:29:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.906+0000 7f1b4e7fc700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b440a13e0 0x7f1b440a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.907+0000 7f1b4e7fc700 1 -- 172.21.15.82:0/3168812710 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1b480ccb00 con 0x7f1b58100850 2023-07-14T19:29:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.907+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b440a13e0 0x7f1b440a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.907+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b440a13e0 0x7f1b440a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400a990 tx=0x7f1b54005c80 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.907+0000 7f1b6b9f3b80 1 Processor -- start 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 -- start start 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ec1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec1790 0x562316ec56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316dc5cb0 con 0x562316ec5c30 2023-07-14T19:29:29.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316e18a40 con 0x562316ec1790 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316e11ed0 con 0x562316ec0e80 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43344/0 (socket says 172.21.15.82:43344) 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/2475231737 learned_addr learned my addr 172.21.15.82:0/2475231737 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ec1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/2475231737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 msgr2=0x562316ec1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ec1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec1790 0x562316ec56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/2475231737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec1790 msgr2=0x562316ec56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec1790 0x562316ec56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/2475231737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562316c3fde0 con 0x562316ec5c30 2023-07-14T19:29:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec1790 0x562316ec56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec6080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009840 tx=0x7f1b50009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284cf53d87be08ea server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.908+0000 7f1b437fe700 1 -- 172.21.15.82:0/2475231737 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000e040 con 0x562316ec5c30 2023-07-14T19:29:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b437fe700 1 -- 172.21.15.82:0/2475231737 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b50003a00 con 0x562316ec5c30 2023-07-14T19:29:29.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b437fe700 1 -- 172.21.15.82:0/2475231737 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50005070 con 0x562316ec5c30 2023-07-14T19:29:29.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2475231737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 msgr2=0x562316ec6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec6080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009840 tx=0x7f1b50009b80 comp rx=0 tx=0).stop 2023-07-14T19:29:29.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2475231737 shutdown_connections 2023-07-14T19:29:29.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec6080 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ec1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2475231737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec1790 0x562316ec56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2475231737 >> 172.21.15.82:0/2475231737 conn(0x562316eb6680 msgr2=0x562316ebbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2475231737 shutdown_connections 2023-07-14T19:29:29.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.909+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2475231737 wait complete. 2023-07-14T19:29:29.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 -- start start 2023-07-14T19:29:29.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 Processor -- start 2023-07-14T19:29:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ecd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316eca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec5c30 0x562316ecae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316db1ac0 con 0x562316ec1790 2023-07-14T19:29:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316db17c0 con 0x562316ec0e80 2023-07-14T19:29:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316c95920 con 0x562316ec5c30 2023-07-14T19:29:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b6b9f3b80 10 client.0 _ll_register_callbacks cb 0x562316daf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b5cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316eca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b5cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316eca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43348/0 (socket says 172.21.15.82:43348) 2023-07-14T19:29:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/1882142251 learned_addr learned my addr 172.21.15.82:0/1882142251 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ecd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.910+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec5c30 0x562316ecae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec5c30 msgr2=0x562316ecae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec5c30 0x562316ecae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/1882142251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 msgr2=0x562316ecd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ecd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/1882142251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5810d300 con 0x562316ec1790 2023-07-14T19:29:29.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ecd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec5c30 0x562316ecae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316eca6c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1b480ca8e0 tx=0x7f1b48099b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8b472263acad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:29.930 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b40ff9700 1 -- 172.21.15.82:0/1882142251 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48099100 con 0x562316ec1790 2023-07-14T19:29:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b40ff9700 1 -- 172.21.15.82:0/1882142251 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1b480a19d0 con 0x562316ec1790 2023-07-14T19:29:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562316d6b4f0 con 0x562316ec1790 2023-07-14T19:29:29.931 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b40ff9700 1 -- 172.21.15.82:0/1882142251 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b480a1b90 con 0x562316ec1790 2023-07-14T19:29:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 10 client.15711 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562316c28c60 con 0x562316ec1790 2023-07-14T19:29:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b40ff9700 1 -- 172.21.15.82:0/1882142251 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562316d6b4f0 con 0x562316ec1790 2023-07-14T19:29:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b40ff9700 1 -- 172.21.15.82:0/1882142251 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b480ca440 con 0x562316ec1790 2023-07-14T19:29:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 10 client.15711 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-14T19:29:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 20 client.15711 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316edaba0 0x562316edcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 4 client.15711 mds_command: new command op to 24481 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.911+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562316e8fda0 con 0x562316edaba0 2023-07-14T19:29:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.912+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316edaba0 0x562316edcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.912+0000 7f1b40ff9700 10 client.15711 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.912+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316edaba0 0x562316edcfb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c8d106010299357 server_cookie=bb80ad590be9743f in_seq=0 out_seq=0 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b40ff9700 1 -- 172.21.15.82:0/1882142251 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562316e8fda0 con 0x562316edaba0 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b40ff9700 10 client.15711 handle_command_reply: tid=0 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 client.15711 shutdown 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 msgr2=0x562316eca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316eca6c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1b480ca8e0 tx=0x7f1b48099b40 comp rx=0 tx=0).stop 2023-07-14T19:29:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 shutdown_connections 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316edaba0 0x562316edcfb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec5c30 0x562316ecae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b40ff9700 0 client.15711 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ecd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/1882142251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316eca6c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 >> 172.21.15.82:0/1882142251 conn(0x562316eb6680 msgr2=0x562316ebba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 shutdown_connections 2023-07-14T19:29:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.913+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/1882142251 wait complete. 2023-07-14T19:29:29.938 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.914+0000 7f1b6b9f3b80 20 client.15711 trim_cache size 0 max 16384 2023-07-14T19:29:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 Processor -- start 2023-07-14T19:29:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 -- start start 2023-07-14T19:29:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ec3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec1790 0x562316ec3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316eda9c0 con 0x562316ec5c30 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316ebc420 con 0x562316ec0e80 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316ed6bc0 con 0x562316ec1790 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b5ddf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b5ddf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43358/0 (socket says 172.21.15.82:43358) 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.926+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/3114083584 learned_addr learned my addr 172.21.15.82:0/3114083584 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ec3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec1790 0x562316ec3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/3114083584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec1790 msgr2=0x562316ec3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec1790 0x562316ec3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/3114083584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 msgr2=0x562316ec3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ec3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5ddf8700 1 -- 172.21.15.82:0/3114083584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562316c66250 con 0x562316ec5c30 2023-07-14T19:29:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec4300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b500060e0 tx=0x7f1b5000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a340cff25b27a699 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b43fff700 1 -- 172.21.15.82:0/3114083584 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000e070 con 0x562316ec5c30 2023-07-14T19:29:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b43fff700 1 -- 172.21.15.82:0/3114083584 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b50003d70 con 0x562316ec5c30 2023-07-14T19:29:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.927+0000 7f1b43fff700 1 -- 172.21.15.82:0/3114083584 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b50005310 con 0x562316ec5c30 2023-07-14T19:29:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/3114083584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 msgr2=0x562316ec4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec4300 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b500060e0 tx=0x7f1b5000b7e0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/3114083584 shutdown_connections 2023-07-14T19:29:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec5c30 0x562316ec4300 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec0e80 0x562316ec3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.946 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/3114083584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec1790 0x562316ec3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/3114083584 >> 172.21.15.82:0/3114083584 conn(0x562316eb6680 msgr2=0x562316ec8010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/3114083584 shutdown_connections 2023-07-14T19:29:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/3114083584 wait complete. 2023-07-14T19:29:29.947 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- start start 2023-07-14T19:29:29.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 Processor -- start 2023-07-14T19:29:29.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ed1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316ecfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.948 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec5c30 0x562316ed0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316ed07c0 con 0x562316ec1790 2023-07-14T19:29:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316ed0900 con 0x562316ec5c30 2023-07-14T19:29:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562316ed0ac0 con 0x562316ec0e80 2023-07-14T19:29:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b6b9f3b80 10 client.0 _ll_register_callbacks cb 0x7f1b480995e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b5cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316ecfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b5cdf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316ecfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43362/0 (socket says 172.21.15.82:43362) 2023-07-14T19:29:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.928+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/2701163349 learned_addr learned my addr 172.21.15.82:0/2701163349 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ed1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5ddf8700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec5c30 0x562316ed0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 msgr2=0x562316ed1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ed1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/2701163349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec5c30 msgr2=0x562316ed0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec5c30 0x562316ed0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5cdf6700 1 -- 172.21.15.82:0/2701163349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562316c3fde0 con 0x562316ec1790 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b5cdf6700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316ecfd40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005950 tx=0x7f1b48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4c810f6a2990d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b40ff9700 1 -- 172.21.15.82:0/2701163349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b480b2200 con 0x562316ec1790 2023-07-14T19:29:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b40ff9700 1 -- 172.21.15.82:0/2701163349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1b480a19d0 con 0x562316ec1790 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b50003a00 con 0x562316ec1790 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b40ff9700 1 -- 172.21.15.82:0/2701163349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b48098a90 con 0x562316ec1790 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b6b9f3b80 10 client.15717 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562316c3e2b0 con 0x562316ec1790 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.929+0000 7f1b40ff9700 1 -- 172.21.15.82:0/2701163349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b50003a00 con 0x562316ec1790 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b40ff9700 1 -- 172.21.15.82:0/2701163349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b48098f00 con 0x562316ec1790 2023-07-14T19:29:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b6b9f3b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b6b9f3b80 10 client.15717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b6b9f3b80 20 client.15717 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316f2f7e0 0x562316f31bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b6b9f3b80 4 client.15717 mds_command: new command op to 24481 tid=0[{"prefix": "damage ls"}] 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 --> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562316e0aeb0 con 0x562316f2f7e0 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316f2f7e0 0x562316f31bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b40ff9700 10 client.15717 ms_handle_connect on v2:172.21.15.161:6838/4007435956 2023-07-14T19:29:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.930+0000 7f1b5d5f7700 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316f2f7e0 0x562316f31bc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ff10451d0e1fa97 server_cookie=39f7b46582ead499 in_seq=0 out_seq=0 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b40ff9700 1 -- 172.21.15.82:0/2701163349 <== mds.2 v2:172.21.15.161:6838/4007435956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562316e0aeb0 con 0x562316f2f7e0 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b40ff9700 10 client.15717 handle_command_reply: tid=0 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 client.15717 shutdown 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 msgr2=0x562316ecfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316ecfd40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005950 tx=0x7f1b48000bc0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 shutdown_connections 2023-07-14T19:29:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562316ec5c30 0x562316ed0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:6838/4007435956,v1:172.21.15.161:6839/4007435956] conn(0x562316f2f7e0 0x562316f31bc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562316ec0e80 0x562316ed1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 --2- 172.21.15.82:0/2701163349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562316ec1790 0x562316ecfd40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 >> 172.21.15.82:0/2701163349 conn(0x562316eb6680 msgr2=0x562316ec3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b40ff9700 0 client.15717 ms_handle_reset on v2:172.21.15.161:6838/4007435956 2023-07-14T19:29:29.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 shutdown_connections 2023-07-14T19:29:29.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 1 -- 172.21.15.82:0/2701163349 wait complete. 2023-07-14T19:29:29.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.931+0000 7f1b6b9f3b80 20 client.15717 trim_cache size 0 max 16384 2023-07-14T19:29:29.956 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-07-14T19:29:29.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b440a13e0 msgr2=0x7f1b440a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b440a13e0 0x7f1b440a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400a990 tx=0x7f1b54005c80 comp rx=0 tx=0).stop 2023-07-14T19:29:29.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 msgr2=0x7f1b58196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:29.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 0x7f1b58196330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800d950 tx=0x7f1b4800dd10 comp rx=0 tx=0).stop 2023-07-14T19:29:29.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 shutdown_connections 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b440a13e0 0x7f1b440a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b58104ef0 0x7f1b5819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b580ffeb0 0x7f1b58195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 --2- 172.21.15.82:0/3168812710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b58100850 0x7f1b58196330 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 >> 172.21.15.82:0/3168812710 conn(0x7f1b580fb760 msgr2=0x7f1b580fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 shutdown_connections 2023-07-14T19:29:29.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:29.932+0000 7f1b5f85b700 1 -- 172.21.15.82:0/3168812710 wait complete. 2023-07-14T19:29:29.960 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-14T19:29:30.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 -- 172.21.15.82:0/3751972227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98600feb50 msgr2=0x7f98600fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.083 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 --2- 172.21.15.82:0/3751972227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98600feb50 0x7f98600fefc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f985c009990 tx=0x7f985c009ca0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 -- 172.21.15.82:0/3751972227 shutdown_connections 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 --2- 172.21.15.82:0/3751972227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98600ff500 0x7f986010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 --2- 172.21.15.82:0/3751972227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9860104b80 0x7f9860104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 --2- 172.21.15.82:0/3751972227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98600feb50 0x7f98600fefc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 -- 172.21.15.82:0/3751972227 >> 172.21.15.82:0/3751972227 conn(0x7f98600fa610 msgr2=0x7f98600fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 -- 172.21.15.82:0/3751972227 shutdown_connections 2023-07-14T19:29:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.081+0000 7f986816f700 1 -- 172.21.15.82:0/3751972227 wait complete. 2023-07-14T19:29:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 Processor -- start 2023-07-14T19:29:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 -- start start 2023-07-14T19:29:30.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98600feb50 0x7f986019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.086 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98600ff500 0x7f986019af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 0x7f9860194af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860195150 con 0x7f9860104b80 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601952c0 con 0x7f98600ff500 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986816f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860195480 con 0x7f98600feb50 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986670c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 0x7f9860194af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986670c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 0x7f9860194af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43378/0 (socket says 172.21.15.82:43378) 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986670c700 1 -- 172.21.15.82:0/2919431424 learned_addr learned my addr 172.21.15.82:0/2919431424 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.087 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.082+0000 7f986570a700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98600ff500 0x7f986019af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98600feb50 0x7f986019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986670c700 1 -- 172.21.15.82:0/2919431424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98600feb50 msgr2=0x7f986019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986670c700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98600feb50 0x7f986019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986670c700 1 -- 172.21.15.82:0/2919431424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98600ff500 msgr2=0x7f986019af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986670c700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98600ff500 0x7f986019af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986670c700 1 -- 172.21.15.82:0/2919431424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0095f0 con 0x7f9860104b80 2023-07-14T19:29:30.088 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98600feb50 0x7f986019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986670c700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 0x7f9860194af0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f985800ef30 tx=0x7f985800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f9856ffd700 1 -- 172.21.15.82:0/2919431424 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858018070 con 0x7f9860104b80 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f9856ffd700 1 -- 172.21.15.82:0/2919431424 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f985800f040 con 0x7f9860104b80 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860195700 con 0x7f9860104b80 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f9856ffd700 1 -- 172.21.15.82:0/2919431424 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800cdb0 con 0x7f9860104b80 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.083+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601a1ac0 con 0x7f9860104b80 2023-07-14T19:29:30.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.085+0000 7f9856ffd700 1 -- 172.21.15.82:0/2919431424 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f9858003a00 con 0x7f9860104b80 2023-07-14T19:29:30.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.086+0000 7f9856ffd700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f984c0a13e0 0x7f984c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.094 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.086+0000 7f9856ffd700 1 -- 172.21.15.82:0/2919431424 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f98580ccaf0 con 0x7f9860104b80 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.086+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f984c0a13e0 0x7f984c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.086+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f984c0a13e0 0x7f984c0a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9850005fd0 tx=0x7f9850005d40 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.086+0000 7f9874307b80 1 Processor -- start 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 -- start start 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff2090 0x562417ff2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 0x562417ff6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff6e40 0x562417ff7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.095 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417ef4cb0 con 0x562417ff29a0 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417f47a40 con 0x562417ff2090 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9874307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417f40ed0 con 0x562417ff6e40 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 0x562417ff6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 0x562417ff6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43384/0 (socket says 172.21.15.82:43384) 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 -- 172.21.15.82:0/4205052909 learned_addr learned my addr 172.21.15.82:0/4205052909 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f9865f0b700 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff2090 0x562417ff2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.096 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986670c700 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff6e40 0x562417ff7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 -- 172.21.15.82:0/4205052909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff6e40 msgr2=0x562417ff7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff6e40 0x562417ff7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 -- 172.21.15.82:0/4205052909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff2090 msgr2=0x562417ff2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff2090 0x562417ff2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 -- 172.21.15.82:0/4205052909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562417d6ede0 con 0x562417ff29a0 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.087+0000 7f986570a700 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 0x562417ff6900 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f985c015000 tx=0x7f985c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9e130aa97187c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.097 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f984bfff700 1 -- 172.21.15.82:0/4205052909 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c01d070 con 0x562417ff29a0 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f984bfff700 1 -- 172.21.15.82:0/4205052909 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f985c01e040 con 0x562417ff29a0 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f984bfff700 1 -- 172.21.15.82:0/4205052909 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c004b20 con 0x562417ff29a0 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 -- 172.21.15.82:0/4205052909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 msgr2=0x562417ff6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 0x562417ff6900 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f985c015000 tx=0x7f985c00b850 comp rx=0 tx=0).stop 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 -- 172.21.15.82:0/4205052909 shutdown_connections 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff6e40 0x562417ff7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff2090 0x562417ff2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.098 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 --2- 172.21.15.82:0/4205052909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff29a0 0x562417ff6900 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 -- 172.21.15.82:0/4205052909 >> 172.21.15.82:0/4205052909 conn(0x562417fe5680 msgr2=0x562417feaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.088+0000 7f9874307b80 1 -- 172.21.15.82:0/4205052909 shutdown_connections 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 -- 172.21.15.82:0/4205052909 wait complete. 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 -- start start 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 Processor -- start 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 0x562417ffe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.099 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff29a0 0x562417ffb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff6e40 0x562417ffbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417ee0ac0 con 0x562417ff2090 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417ee07c0 con 0x562417ff6e40 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417dc4920 con 0x562417ff29a0 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9874307b80 10 client.0 _ll_register_callbacks cb 0x562417ede760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9865f0b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 0x562417ffe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9865f0b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 0x562417ffe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43388/0 (socket says 172.21.15.82:43388) 2023-07-14T19:29:30.100 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.089+0000 7f9865f0b700 1 -- 172.21.15.82:0/2377831687 learned_addr learned my addr 172.21.15.82:0/2377831687 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f986670c700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff6e40 0x562417ffbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f986570a700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff29a0 0x562417ffb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9865f0b700 1 -- 172.21.15.82:0/2377831687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff29a0 msgr2=0x562417ffb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff29a0 0x562417ffb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9865f0b700 1 -- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff6e40 msgr2=0x562417ffbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff6e40 0x562417ffbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9865f0b700 1 -- 172.21.15.82:0/2377831687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986010c240 con 0x562417ff2090 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f986570a700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff29a0 0x562417ffb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9865f0b700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 0x562417ffe610 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f985000f9d0 tx=0x7f9850004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29a0ae9741f6863 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f98497fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f98497fa700 1 -- 172.21.15.82:0/2377831687 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985001d640 con 0x562417ff2090 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562417e9a4f0 con 0x562417ff2090 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f98497fa700 1 -- 172.21.15.82:0/2377831687 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f985001d7a0 con 0x562417ff2090 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f98497fa700 1 -- 172.21.15.82:0/2377831687 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f985001d960 con 0x562417ff2090 2023-07-14T19:29:30.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9874307b80 10 client.15729 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562417d57c60 con 0x562417ff2090 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.090+0000 7f98497fa700 1 -- 172.21.15.82:0/2377831687 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562417e9a4f0 con 0x562417ff2090 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f98497fa700 1 -- 172.21.15.82:0/2377831687 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9850030c80 con 0x562417ff2090 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f9874307b80 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f9874307b80 10 client.15729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f9874307b80 20 client.15729 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:30.103 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f9874307b80 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x56241800be40 0x56241800e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f9874307b80 4 client.15729 mds_command: new command op to 24491 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562417fbeda0 con 0x56241800be40 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f986570a700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x56241800be40 0x56241800e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f98497fa700 10 client.15729 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.091+0000 7f986570a700 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x56241800be40 0x56241800e250 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b11575e72b95cb4 server_cookie=23b6789e32785f9a in_seq=0 out_seq=0 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f98497fa700 1 -- 172.21.15.82:0/2377831687 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x562417fbeda0 con 0x56241800be40 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f98497fa700 10 client.15729 handle_command_reply: tid=0 2023-07-14T19:29:30.104 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 client.15729 shutdown 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 msgr2=0x562417ffe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 0x562417ffe610 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f985000f9d0 tx=0x7f9850004750 comp rx=0 tx=0).stop 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 shutdown_connections 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x56241800be40 0x56241800e250 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff6e40 0x562417ffbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f98497fa700 0 client.15729 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:29:30.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff2090 0x562417ffe610 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 --2- 172.21.15.82:0/2377831687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff29a0 0x562417ffb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 >> 172.21.15.82:0/2377831687 conn(0x562417fe5680 msgr2=0x562417feaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 shutdown_connections 2023-07-14T19:29:30.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.092+0000 7f9874307b80 1 -- 172.21.15.82:0/2377831687 wait complete. 2023-07-14T19:29:30.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.093+0000 7f9874307b80 20 client.15729 trim_cache size 0 max 16384 2023-07-14T19:29:30.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 Processor -- start 2023-07-14T19:29:30.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 -- start start 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x562417ffe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562417ffe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562417ff9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417fec2d0 con 0x562417ff6e40 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417ff2f30 con 0x562417ff29a0 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9874307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562417ff3150 con 0x562417ff2090 2023-07-14T19:29:30.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562417ff9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562417ff9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43400/0 (socket says 172.21.15.82:43400) 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 -- 172.21.15.82:0/115251682 learned_addr learned my addr 172.21.15.82:0/115251682 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 -- 172.21.15.82:0/115251682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 msgr2=0x562417ffe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986570a700 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562417ffe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x562417ffe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 -- 172.21.15.82:0/115251682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 msgr2=0x562417ffe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562417ffe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.109 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 -- 172.21.15.82:0/115251682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562417d95250 con 0x562417ff6e40 2023-07-14T19:29:30.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f986670c700 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562417ff9220 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9858099a20 tx=0x7f9858099950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c2a027d83a90c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9854ff9700 1 -- 172.21.15.82:0/115251682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580b2220 con 0x562417ff6e40 2023-07-14T19:29:30.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9854ff9700 1 -- 172.21.15.82:0/115251682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9858099a90 con 0x562417ff6e40 2023-07-14T19:29:30.110 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.106+0000 7f9854ff9700 1 -- 172.21.15.82:0/115251682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858099f50 con 0x562417ff6e40 2023-07-14T19:29:30.113 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 -- 172.21.15.82:0/115251682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 msgr2=0x562417ff9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562417ff9220 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9858099a20 tx=0x7f9858099950 comp rx=0 tx=0).stop 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 -- 172.21.15.82:0/115251682 shutdown_connections 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562417ff9220 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x562417ffe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 --2- 172.21.15.82:0/115251682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562417ffe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 -- 172.21.15.82:0/115251682 >> 172.21.15.82:0/115251682 conn(0x562417fe5680 msgr2=0x562417ff4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 -- 172.21.15.82:0/115251682 shutdown_connections 2023-07-14T19:29:30.114 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 -- 172.21.15.82:0/115251682 wait complete. 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 -- start start 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 Processor -- start 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x5624180027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.107+0000 7f9874307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562418002d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562418000ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624180015d0 con 0x562417ff6e40 2023-07-14T19:29:30.115 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418001740 con 0x562417ff29a0 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562418001900 con 0x562417ff2090 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 10 client.0 _ll_register_callbacks cb 0x7f985000dd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562418000ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562418000ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43406/0 (socket says 172.21.15.82:43406) 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 -- 172.21.15.82:0/3196014717 learned_addr learned my addr 172.21.15.82:0/3196014717 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986570a700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562418002d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9865f0b700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x5624180027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.116 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 -- 172.21.15.82:0/3196014717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 msgr2=0x5624180027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x5624180027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 -- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 msgr2=0x562418002d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562418002d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 -- 172.21.15.82:0/3196014717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601a1ac0 con 0x562417ff6e40 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9865f0b700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x5624180027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f986670c700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562418000ec0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9858099a20 tx=0x7f98580b2220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51c9108d87d031a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.117 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f98497fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f98497fa700 1 -- 172.21.15.82:0/3196014717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98580a1e50 con 0x562417ff6e40 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f98497fa700 1 -- 172.21.15.82:0/3196014717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9858099cf0 con 0x562417ff6e40 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f985c01e040 con 0x562417ff6e40 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f98497fa700 1 -- 172.21.15.82:0/3196014717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9858099eb0 con 0x562417ff6e40 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 10 client.15735 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562417d6d2b0 con 0x562417ff6e40 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.108+0000 7f98497fa700 1 -- 172.21.15.82:0/3196014717 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9858063080 con 0x562417ff6e40 2023-07-14T19:29:30.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f98497fa700 1 -- 172.21.15.82:0/3196014717 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f985801b9d0 con 0x562417ff6e40 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9874307b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9874307b80 10 client.15735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9874307b80 20 client.15735 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9874307b80 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x562418060820 0x562418062c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9874307b80 4 client.15735 mds_command: new command op to 24491 tid=0[{"prefix": "damage ls"}] 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 --> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562417f39eb0 con 0x562418060820 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9865f0b700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x562418060820 0x562418062c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.119 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f98497fa700 10 client.15735 ms_handle_connect on v2:172.21.15.133:6836/46070579 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.109+0000 7f9865f0b700 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x562418060820 0x562418062c00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8aa526e996a915c1 server_cookie=3858b8de304008ea in_seq=0 out_seq=0 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f98497fa700 1 -- 172.21.15.82:0/3196014717 <== mds.1 v2:172.21.15.133:6836/46070579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+528 (crc 0 0 0) 0x562417f39eb0 con 0x562418060820 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f98497fa700 10 client.15735 handle_command_reply: tid=0 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 client.15735 shutdown 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 msgr2=0x562418000ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562418000ec0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9858099a20 tx=0x7f98580b2220 comp rx=0 tx=0).stop 2023-07-14T19:29:30.120 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 shutdown_connections 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:6836/46070579,v1:172.21.15.133:6837/46070579] conn(0x562418060820 0x562418062c00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562417ff6e40 0x562418000ec0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562417ff2090 0x5624180027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 --2- 172.21.15.82:0/3196014717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562417ff29a0 0x562418002d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f98497fa700 0 client.15735 ms_handle_reset on v2:172.21.15.133:6836/46070579 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 >> 172.21.15.82:0/3196014717 conn(0x562417fe5680 msgr2=0x562417ff9220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 shutdown_connections 2023-07-14T19:29:30.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 1 -- 172.21.15.82:0/3196014717 wait complete. 2023-07-14T19:29:30.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f9874307b80 20 client.15735 trim_cache size 0 max 16384 2023-07-14T19:29:30.122 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2023-07-14T19:29:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:29:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2023-07-14T19:29:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 150246500, 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511628283, 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505" 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 267719169, 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627781, 2023-07-14T19:29:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/client.0" 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 1304816674, 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511628284, 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/volumes/_nogroup/sv_1/.meta" 2023-07-14T19:29:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:29:30.125 INFO:teuthology.orchestra.run.smithi082.stdout:] 2023-07-14T19:29:30.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f984c0a13e0 msgr2=0x7f984c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.125 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f984c0a13e0 0x7f984c0a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9850005fd0 tx=0x7f9850005d40 comp rx=0 tx=0).stop 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 msgr2=0x7f9860194af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 0x7f9860194af0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f985800ef30 tx=0x7f985800c3a0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 shutdown_connections 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f984c0a13e0 0x7f984c0a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9860104b80 0x7f9860194af0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98600feb50 0x7f986019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.126 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 --2- 172.21.15.82:0/2919431424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98600ff500 0x7f986019af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 >> 172.21.15.82:0/2919431424 conn(0x7f98600fa610 msgr2=0x7f98600fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 shutdown_connections 2023-07-14T19:29:30.127 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.110+0000 7f986816f700 1 -- 172.21.15.82:0/2919431424 wait complete. 2023-07-14T19:29:30.127 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:29 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1101108672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:29:30.128 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-14T19:29:30.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:29 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1101108672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:29:30.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:29 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1101108672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:29:30.252 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.249+0000 7f9cbb60f700 1 -- 172.21.15.82:0/2057090677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb410a1a0 msgr2=0x7f9cb410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.249+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/2057090677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb410a1a0 0x7f9cb410c5e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00b0a0 tx=0x7f9cac00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 -- 172.21.15.82:0/2057090677 shutdown_connections 2023-07-14T19:29:30.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/2057090677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb410a1a0 0x7f9cb410c5e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/2057090677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 0x7f9cb41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/2057090677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb41016e0 0x7f9cb4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.253 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 -- 172.21.15.82:0/2057090677 >> 172.21.15.82:0/2057090677 conn(0x7f9cb40fa5e0 msgr2=0x7f9cb40fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 -- 172.21.15.82:0/2057090677 shutdown_connections 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 -- 172.21.15.82:0/2057090677 wait complete. 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 Processor -- start 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 -- start start 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 0x7f9cb4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 0x7f9cb4196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.254 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.250+0000 7f9cbb60f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb410a1a0 0x7f9cb419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cbb60f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb419ae00 con 0x7f9cb41016e0 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cbb60f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb419af70 con 0x7f9cb410a1a0 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cbb60f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb419b130 con 0x7f9cb4100dd0 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 0x7f9cb4196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 0x7f9cb4196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43416/0 (socket says 172.21.15.82:43416) 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 -- 172.21.15.82:0/1003573835 learned_addr learned my addr 172.21.15.82:0/1003573835 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb410a1a0 0x7f9cb419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.255 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 0x7f9cb4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 -- 172.21.15.82:0/1003573835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 msgr2=0x7f9cb4195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 0x7f9cb4195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 -- 172.21.15.82:0/1003573835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb410a1a0 msgr2=0x7f9cb419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb410a1a0 0x7f9cb419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 -- 172.21.15.82:0/1003573835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac009d00 con 0x7f9cb41016e0 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 0x7f9cb4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 0x7f9cb4196360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400e9a0 tx=0x7f9ca400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.256 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9caa7fc700 1 -- 172.21.15.82:0/1003573835 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4009790 con 0x7f9cb41016e0 2023-07-14T19:29:30.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9caa7fc700 1 -- 172.21.15.82:0/1003573835 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca400cbd0 con 0x7f9cb41016e0 2023-07-14T19:29:30.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9caa7fc700 1 -- 172.21.15.82:0/1003573835 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4018680 con 0x7f9cb41016e0 2023-07-14T19:29:30.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb419b410 con 0x7f9cb41016e0 2023-07-14T19:29:30.257 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.251+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb41a2c50 con 0x7f9cb41016e0 2023-07-14T19:29:30.262 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.253+0000 7f9caa7fc700 1 -- 172.21.15.82:0/1003573835 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f9ca400cd40 con 0x7f9cb41016e0 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.254+0000 7f9caa7fc700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9ca00a13e0 0x7f9ca00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.254+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9ca00a13e0 0x7f9ca00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.254+0000 7f9caa7fc700 1 -- 172.21.15.82:0/1003573835 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9ca4014070 con 0x7f9cb41016e0 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.254+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9ca00a13e0 0x7f9ca00a3890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0005d90 tx=0x7f9cb0005d00 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 Processor -- start 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 -- start start 2023-07-14T19:29:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d69e80 0x55c755d6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d6a790 0x55c755d6e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 0x55c755d6f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755c6ecb0 con 0x55c755d6ec30 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755cc1a40 con 0x55c755d6a790 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755cbaed0 con 0x55c755d69e80 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 0x55c755d6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 0x55c755d6f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43426/0 (socket says 172.21.15.82:43426) 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1283479799 learned_addr learned my addr 172.21.15.82:0/1283479799 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d6a790 0x55c755d6e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d69e80 0x55c755d6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1283479799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d69e80 msgr2=0x55c755d6a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d69e80 0x55c755d6a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1283479799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d6a790 msgr2=0x55c755d6e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.255+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d6a790 0x55c755d6e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.265 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1283479799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c755ae8de0 con 0x55c755d6ec30 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 0x55c755d6f080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009310 tx=0x7f9cac009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22024843ffa455d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9c9f7fe700 1 -- 172.21.15.82:0/1283479799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00e040 con 0x55c755d6ec30 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9c9f7fe700 1 -- 172.21.15.82:0/1283479799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cac00bcb0 con 0x55c755d6ec30 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9c9f7fe700 1 -- 172.21.15.82:0/1283479799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac01c640 con 0x55c755d6ec30 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1283479799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 msgr2=0x55c755d6f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 0x55c755d6f080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009310 tx=0x7f9cac009340 comp rx=0 tx=0).stop 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.256+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1283479799 shutdown_connections 2023-07-14T19:29:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6ec30 0x55c755d6f080 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d69e80 0x55c755d6a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1283479799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d6a790 0x55c755d6e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1283479799 >> 172.21.15.82:0/1283479799 conn(0x55c755d5f680 msgr2=0x55c755d64ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1283479799 shutdown_connections 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1283479799 wait complete. 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- start start 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 Processor -- start 2023-07-14T19:29:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d76380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d73600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d6ec30 0x55c755d73c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755c5aac0 con 0x55c755d6a790 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755c5a7c0 con 0x55c755d69e80 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755b3e920 con 0x55c755d6ec30 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cc77a7b80 10 client.0 _ll_register_callbacks cb 0x55c755c58760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.257+0000 7f9cb8baa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d73600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d73600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43430/0 (socket says 172.21.15.82:43430) 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 -- 172.21.15.82:0/2644287461 learned_addr learned my addr 172.21.15.82:0/2644287461 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d76380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d6ec30 0x55c755d73c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 -- 172.21.15.82:0/2644287461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d6ec30 msgr2=0x55c755d73c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d6ec30 0x55c755d73c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 -- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 msgr2=0x55c755d76380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.269 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d76380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 -- 172.21.15.82:0/2644287461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb410d2a0 con 0x55c755d6a790 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d6ec30 0x55c755d73c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d73600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4099f80 tx=0x7f9ca409a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7af38f982d19259 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9c9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9c9cff9700 1 -- 172.21.15.82:0/2644287461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40b1070 con 0x55c755d6a790 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9c9cff9700 1 -- 172.21.15.82:0/2644287461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9ca40a2490 con 0x55c755d6a790 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c755c144f0 con 0x55c755d6a790 2023-07-14T19:29:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9c9cff9700 1 -- 172.21.15.82:0/2644287461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca40a2210 con 0x55c755d6a790 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cc77a7b80 10 client.15747 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c755ad1c60 con 0x55c755d6a790 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.258+0000 7f9c9cff9700 1 -- 172.21.15.82:0/2644287461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca40c8080 con 0x55c755d6a790 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9c9cff9700 1 -- 172.21.15.82:0/2644287461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9ca4098fa0 con 0x55c755d6a790 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cc77a7b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cc77a7b80 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cc77a7b80 20 client.15747 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d83bc0 0x55c755d85fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cc77a7b80 4 client.15747 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c755d38da0 con 0x55c755d83bc0 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d83bc0 0x55c755d85fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9c9cff9700 10 client.15747 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.259+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d83bc0 0x55c755d85fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4654328b6a6cf35 server_cookie=ad218b6cb60d91a4 in_seq=0 out_seq=0 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9c9cff9700 1 -- 172.21.15.82:0/2644287461 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c755d38da0 con 0x55c755d83bc0 2023-07-14T19:29:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9c9cff9700 10 client.15747 handle_command_reply: tid=0 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 client.15747 shutdown 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 msgr2=0x55c755d73600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d73600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4099f80 tx=0x7f9ca409a580 comp rx=0 tx=0).stop 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 shutdown_connections 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d83bc0 0x55c755d85fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d6ec30 0x55c755d73c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9c9cff9700 0 client.15747 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d76380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/2644287461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d73600 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 >> 172.21.15.82:0/2644287461 conn(0x55c755d5f680 msgr2=0x55c755d64a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 shutdown_connections 2023-07-14T19:29:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/2644287461 wait complete. 2023-07-14T19:29:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.260+0000 7f9cc77a7b80 20 client.15747 trim_cache size 0 max 16384 2023-07-14T19:29:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 Processor -- start 2023-07-14T19:29:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- start start 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755d7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d6cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d6d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755d80660 con 0x55c755d6a790 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755d812b0 con 0x55c755d69e80 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755d66790 con 0x55c755d5f680 2023-07-14T19:29:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d6d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d6d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43440/0 (socket says 172.21.15.82:43440) 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 -- 172.21.15.82:0/3459835668 learned_addr learned my addr 172.21.15.82:0/3459835668 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755d7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d6cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 -- 172.21.15.82:0/3459835668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 msgr2=0x55c755d7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755d7aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 -- 172.21.15.82:0/3459835668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 msgr2=0x55c755d6cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d6cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 -- 172.21.15.82:0/3459835668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c755d60e20 con 0x55c755d6a790 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d6cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d6d240 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9cac021000 tx=0x7f9cac006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886389ef5c5d8387 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9c9ffff700 1 -- 172.21.15.82:0/3459835668 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00e040 con 0x55c755d6a790 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9c9ffff700 1 -- 172.21.15.82:0/3459835668 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cac0074a0 con 0x55c755d6a790 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9c9ffff700 1 -- 172.21.15.82:0/3459835668 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0054b0 con 0x55c755d6a790 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/3459835668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 msgr2=0x55c755d6d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.281 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d6d240 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9cac021000 tx=0x7f9cac006000 comp rx=0 tx=0).stop 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/3459835668 shutdown_connections 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755d6d240 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755d7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/3459835668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755d6cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/3459835668 >> 172.21.15.82:0/3459835668 conn(0x55c755d80290 msgr2=0x55c755d71010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/3459835668 shutdown_connections 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.274+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/3459835668 wait complete. 2023-07-14T19:29:30.282 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 -- start start 2023-07-14T19:29:30.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 Processor -- start 2023-07-14T19:29:30.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755dd4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755dd5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.283 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755dd3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755dd4220 con 0x55c755d6a790 2023-07-14T19:29:30.284 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755dd4390 con 0x55c755d69e80 2023-07-14T19:29:30.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c755dd4550 con 0x55c755d5f680 2023-07-14T19:29:30.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cc77a7b80 10 client.0 _ll_register_callbacks cb 0x7f9ca40999c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:30.285 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cb9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755dd3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cb9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755dd3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43446/0 (socket says 172.21.15.82:43446) 2023-07-14T19:29:30.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1275055853 learned_addr learned my addr 172.21.15.82:0/1275055853 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:30.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.275+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755dd4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb8baa700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755dd5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.286 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1275055853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 msgr2=0x55c755dd4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755dd4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 msgr2=0x55c755dd5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:30.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755dd5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb9bac700 1 -- 172.21.15.82:0/1275055853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c755ae8de0 con 0x55c755d6a790 2023-07-14T19:29:30.287 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cb9bac700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755dd3cb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00d010 tx=0x7f9cac003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1fd27ec9ecf5ef server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:30.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9c9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:30.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9c9cff9700 1 -- 172.21.15.82:0/1275055853 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0052c0 con 0x55c755d6a790 2023-07-14T19:29:30.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cac00bcb0 con 0x55c755d6a790 2023-07-14T19:29:30.288 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9c9cff9700 1 -- 172.21.15.82:0/1275055853 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f9cac005420 con 0x55c755d6a790 2023-07-14T19:29:30.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9c9cff9700 1 -- 172.21.15.82:0/1275055853 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cac0055e0 con 0x55c755d6a790 2023-07-14T19:29:30.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cc77a7b80 10 client.15753 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:30.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c755b0f250 con 0x55c755d6a790 2023-07-14T19:29:30.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.276+0000 7f9c9cff9700 1 -- 172.21.15.82:0/1275055853 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cac00bcb0 con 0x55c755d6a790 2023-07-14T19:29:30.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9c9cff9700 1 -- 172.21.15.82:0/1275055853 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cac035030 con 0x55c755d6a790 2023-07-14T19:29:30.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cc77a7b80 10 client.15753 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:30.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cc77a7b80 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:30.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cc77a7b80 20 client.15753 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:30.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d6ec30 0x55c755ddea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:30.290 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cc77a7b80 4 client.15753 mds_command: new command op to 24524 tid=0[{"prefix": "damage ls"}] 2023-07-14T19:29:30.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c755cb3eb0 con 0x55c755d6ec30 2023-07-14T19:29:30.291 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d6ec30 0x55c755ddea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:30.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9c9cff9700 10 client.15753 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:30.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.277+0000 7f9cb93ab700 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d6ec30 0x55c755ddea80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c796d077e2190a server_cookie=2b5be0b08c89f44b in_seq=0 out_seq=0 2023-07-14T19:29:30.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9c9cff9700 1 -- 172.21.15.82:0/1275055853 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c755cb3eb0 con 0x55c755d6ec30 2023-07-14T19:29:30.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9c9cff9700 10 client.15753 handle_command_reply: tid=0 2023-07-14T19:29:30.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 client.15753 shutdown 2023-07-14T19:29:30.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 msgr2=0x55c755dd3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755dd3cb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00d010 tx=0x7f9cac003c30 comp rx=0 tx=0).stop 2023-07-14T19:29:30.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 shutdown_connections 2023-07-14T19:29:30.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c755d6a790 0x55c755dd3cb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c755d6ec30 0x55c755ddea80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c755d5f680 0x55c755dd4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 --2- 172.21.15.82:0/1275055853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c755d69e80 0x55c755dd5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9c9cff9700 0 client.15753 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:30.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 >> 172.21.15.82:0/1275055853 conn(0x55c755d80290 msgr2=0x55c755d71010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 shutdown_connections 2023-07-14T19:29:30.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 1 -- 172.21.15.82:0/1275055853 wait complete. 2023-07-14T19:29:30.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cc77a7b80 20 client.15753 trim_cache size 0 max 16384 2023-07-14T19:29:30.298 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2023-07-14T19:29:30.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9ca00a13e0 msgr2=0x7f9ca00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9ca00a13e0 0x7f9ca00a3890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0005d90 tx=0x7f9cb0005d00 comp rx=0 tx=0).stop 2023-07-14T19:29:30.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 msgr2=0x7f9cb4196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:30.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 0x7f9cb4196360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400e9a0 tx=0x7f9ca400ecb0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 shutdown_connections 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9ca00a13e0 0x7f9ca00a3890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cb410a1a0 0x7f9cb419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9cb4100dd0 0x7f9cb4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 --2- 172.21.15.82:0/1003573835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cb41016e0 0x7f9cb4196360 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.278+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 >> 172.21.15.82:0/1003573835 conn(0x7f9cb40fa5e0 msgr2=0x7f9cb410c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.279+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 shutdown_connections 2023-07-14T19:29:30.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:29:30.279+0000 7f9cbb60f700 1 -- 172.21.15.82:0/1003573835 wait complete. 2023-07-14T19:29:30.301 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 150246500, 'ino': 1099511628283, 'path': '/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505'}, {'damage_type': 'backtrace', 'id': 267719169, 'ino': 1099511627781, 'path': '/volumes/_nogroup/sv_1/6e5f72af-0c6b-494d-b56b-ad34fb765505/client.0'}, {'damage_type': 'backtrace', 'id': 1304816674, 'ino': 1099511628284, 'path': '/volumes/_nogroup/sv_1/.meta'}] 2023-07-14T19:29:30.301 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-14T19:29:31.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:30 smithi133 ceph-mon[128894]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 574 B/s wr, 7 op/s 2023-07-14T19:29:31.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:30 smithi082 ceph-mon[127461]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 574 B/s wr, 7 op/s 2023-07-14T19:29:31.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:30 smithi161 ceph-mon[129184]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 574 B/s wr, 7 op/s 2023-07-14T19:29:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 -- 172.21.15.82:0/1854348936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c069330 msgr2=0x7f055c10ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1854348936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c069330 0x7f055c10ca70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f055400b0d0 tx=0x7f055400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 -- 172.21.15.82:0/1854348936 shutdown_connections 2023-07-14T19:29:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1854348936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c069330 0x7f055c10ca70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1854348936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c105cd0 0x7f055c1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1854348936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f055c068980 0x7f055c068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 -- 172.21.15.82:0/1854348936 >> 172.21.15.82:0/1854348936 conn(0x7f055c076ae0 msgr2=0x7f055c076ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 -- 172.21.15.82:0/1854348936 shutdown_connections 2023-07-14T19:29:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.253+0000 7f0563c8f700 1 -- 172.21.15.82:0/1854348936 wait complete. 2023-07-14T19:29:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 Processor -- start 2023-07-14T19:29:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 -- start start 2023-07-14T19:29:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c068980 0x7f055c0feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f055c069330 0x7f055c0ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 0x7f055c1a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1a4590 con 0x7f055c105cd0 2023-07-14T19:29:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1a46d0 con 0x7f055c069330 2023-07-14T19:29:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f0563c8f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1a4840 con 0x7f055c068980 2023-07-14T19:29:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f056222c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 0x7f055c1a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f056222c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 0x7f055c1a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43460/0 (socket says 172.21.15.82:43460) 2023-07-14T19:29:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.254+0000 7f056222c700 1 -- 172.21.15.82:0/1438432958 learned_addr learned my addr 172.21.15.82:0/1438432958 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c068980 0x7f055c0feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056122a700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f055c069330 0x7f055c0ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056222c700 1 -- 172.21.15.82:0/1438432958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c068980 msgr2=0x7f055c0feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056222c700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c068980 0x7f055c0feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056222c700 1 -- 172.21.15.82:0/1438432958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f055c069330 msgr2=0x7f055c0ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056222c700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f055c069330 0x7f055c0ff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056222c700 1 -- 172.21.15.82:0/1438432958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554009d00 con 0x7f055c105cd0 2023-07-14T19:29:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f056222c700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 0x7f055c1a4050 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f05540077f0 tx=0x7f05540078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f0552ffd700 1 -- 172.21.15.82:0/1438432958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055400e040 con 0x7f055c105cd0 2023-07-14T19:29:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c1a4ac0 con 0x7f055c105cd0 2023-07-14T19:29:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f0552ffd700 1 -- 172.21.15.82:0/1438432958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0554003e10 con 0x7f055c105cd0 2023-07-14T19:29:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f0552ffd700 1 -- 172.21.15.82:0/1438432958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055401b9d0 con 0x7f055c105cd0 2023-07-14T19:29:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.255+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c1a4ee0 con 0x7f055c105cd0 2023-07-14T19:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.257+0000 7f0552ffd700 1 -- 172.21.15.82:0/1438432958 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f05540056a0 con 0x7f055c105cd0 2023-07-14T19:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.258+0000 7f0552ffd700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f05480a13e0 0x7f05480a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.258+0000 7f0552ffd700 1 -- 172.21.15.82:0/1438432958 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f05540ce700 con 0x7f055c105cd0 2023-07-14T19:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.258+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f05480a13e0 0x7f05480a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.258+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f05480a13e0 0x7f05480a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0558005fd0 tx=0x7f0558005db0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 Processor -- start 2023-07-14T19:29:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 -- start start 2023-07-14T19:29:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584227f46b0 0x55842299e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 0x558422998f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842289dc50 con 0x55842299ef00 2023-07-14T19:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584228f09e0 con 0x55842299ea90 2023-07-14T19:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056fe27b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584228e9e70 con 0x5584227f46b0 2023-07-14T19:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056222c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 0x558422998f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056222c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 0x558422998f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43468/0 (socket says 172.21.15.82:43468) 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f056222c700 1 -- 172.21.15.82:0/2115754716 learned_addr learned my addr 172.21.15.82:0/2115754716 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.259+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584227f46b0 0x55842299e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056122a700 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056222c700 1 -- 172.21.15.82:0/2115754716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584227f46b0 msgr2=0x55842299e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056222c700 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584227f46b0 0x55842299e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056222c700 1 -- 172.21.15.82:0/2115754716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 msgr2=0x5584229989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056222c700 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056222c700 1 -- 172.21.15.82:0/2115754716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558422717d80 con 0x55842299ef00 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f056222c700 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 0x558422998f80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f05540079f0 tx=0x7f055409b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ceff5c068330bf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f0547fff700 1 -- 172.21.15.82:0/2115754716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554084ea0 con 0x55842299ef00 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f0547fff700 1 -- 172.21.15.82:0/2115754716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f055409f6d0 con 0x55842299ef00 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.260+0000 7f0547fff700 1 -- 172.21.15.82:0/2115754716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554086310 con 0x55842299ef00 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 -- 172.21.15.82:0/2115754716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 msgr2=0x558422998f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 0x558422998f80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f05540079f0 tx=0x7f055409b4f0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 -- 172.21.15.82:0/2115754716 shutdown_connections 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ef00 0x558422998f80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5584227f46b0 0x55842299e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2115754716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 -- 172.21.15.82:0/2115754716 >> 172.21.15.82:0/2115754716 conn(0x55842298e3d0 msgr2=0x558422993530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 -- 172.21.15.82:0/2115754716 shutdown_connections 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 -- 172.21.15.82:0/2115754716 wait complete. 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 -- start start 2023-07-14T19:29:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.261+0000 7f056fe27b80 1 Processor -- start 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x5584229a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558422889a60 con 0x5584227f46b0 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558422889760 con 0x55842299ea90 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55842276d8c0 con 0x55842299ef00 2023-07-14T19:29:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056fe27b80 10 client.0 _ll_register_callbacks cb 0x558422887700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x5584229a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x5584229a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43470/0 (socket says 172.21.15.82:43470) 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 -- 172.21.15.82:0/2771040984 learned_addr learned my addr 172.21.15.82:0/2771040984 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056122a700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f056222c700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 -- 172.21.15.82:0/2771040984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 msgr2=0x5584229a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 -- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 msgr2=0x5584229a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 -- 172.21.15.82:0/2771040984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c1a4ee0 con 0x5584227f46b0 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x5584229a7ae0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0558010790 tx=0x7f0558004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e002bc902764c97 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f05457fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.262+0000 7f05457fa700 1 -- 172.21.15.82:0/2771040984 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055800e840 con 0x5584227f46b0 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f05457fa700 1 -- 172.21.15.82:0/2771040984 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f055801e840 con 0x5584227f46b0 2023-07-14T19:29:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558422843490 con 0x5584227f46b0 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f05457fa700 1 -- 172.21.15.82:0/2771040984 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f055801ea00 con 0x5584227f46b0 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 10 client.15765 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558422700c00 con 0x5584227f46b0 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f05457fa700 1 -- 172.21.15.82:0/2771040984 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558422843490 con 0x5584227f46b0 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f05457fa700 1 -- 172.21.15.82:0/2771040984 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0558024020 con 0x5584227f46b0 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 20 client.15765 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5584229b2020 0x5584229b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 4 client.15765 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558422967d40 con 0x5584229b2020 2023-07-14T19:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.263+0000 7f056122a700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5584229b2020 0x5584229b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.264+0000 7f05457fa700 10 client.15765 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.264+0000 7f056122a700 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5584229b2020 0x5584229b4430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18308a27d4bac207 server_cookie=4b7e58e07a01e66e in_seq=0 out_seq=0 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.264+0000 7f05457fa700 1 -- 172.21.15.82:0/2771040984 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558422967d40 con 0x5584229b2020 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.264+0000 7f05457fa700 10 client.15765 handle_command_reply: tid=0 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.264+0000 7f056fe27b80 1 client.15765 shutdown 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 msgr2=0x5584229a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x5584229a7ae0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0558010790 tx=0x7f0558004750 comp rx=0 tx=0).stop 2023-07-14T19:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 shutdown_connections 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5584229b2020 0x5584229b4430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x5584229a7ae0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f05457fa700 0 client.15765 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2771040984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 >> 172.21.15.82:0/2771040984 conn(0x55842298e3d0 msgr2=0x558422992c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 shutdown_connections 2023-07-14T19:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 1 -- 172.21.15.82:0/2771040984 wait complete. 2023-07-14T19:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.265+0000 7f056fe27b80 20 client.15765 trim_cache size 0 max 16384 2023-07-14T19:29:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.279+0000 7f056fe27b80 1 Processor -- start 2023-07-14T19:29:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.279+0000 7f056fe27b80 1 -- start start 2023-07-14T19:29:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x55842299dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056fe27b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558422990470 con 0x5584227f46b0 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056fe27b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558422995020 con 0x55842299ea90 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056fe27b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558422994580 con 0x55842299ef00 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x55842299dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x55842299dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43478/0 (socket says 172.21.15.82:43478) 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 -- 172.21.15.82:0/2189899181 learned_addr learned my addr 172.21.15.82:0/2189899181 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056222c700 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f056122a700 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 -- 172.21.15.82:0/2189899181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 msgr2=0x5584229933b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229933b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 -- 172.21.15.82:0/2189899181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 msgr2=0x5584229a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 -- 172.21.15.82:0/2189899181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55842273e1f0 con 0x5584227f46b0 2023-07-14T19:29:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.280+0000 7f0561a2b700 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x55842299dde0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0558026400 tx=0x7f0558005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c94aec35887308 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f0550daa700 1 -- 172.21.15.82:0/2189899181 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558030440 con 0x5584227f46b0 2023-07-14T19:29:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f0550daa700 1 -- 172.21.15.82:0/2189899181 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0558030a60 con 0x5584227f46b0 2023-07-14T19:29:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f0550daa700 1 -- 172.21.15.82:0/2189899181 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055801e640 con 0x5584227f46b0 2023-07-14T19:29:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 -- 172.21.15.82:0/2189899181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 msgr2=0x55842299dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x55842299dde0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0558026400 tx=0x7f0558005cd0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 -- 172.21.15.82:0/2189899181 shutdown_connections 2023-07-14T19:29:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584227f46b0 0x55842299dde0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 --2- 172.21.15.82:0/2189899181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55842299ea90 0x5584229a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 -- 172.21.15.82:0/2189899181 >> 172.21.15.82:0/2189899181 conn(0x55842298e3d0 msgr2=0x5584229a15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 -- 172.21.15.82:0/2189899181 shutdown_connections 2023-07-14T19:29:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.281+0000 7f056fe27b80 1 -- 172.21.15.82:0/2189899181 wait complete. 2023-07-14T19:29:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 -- start start 2023-07-14T19:29:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 Processor -- start 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584227f46b0 0x5584229f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 0x5584229fa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584229ff250 con 0x55842299ea90 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584229ff3c0 con 0x5584227f46b0 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584229ff580 con 0x55842299ef00 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056fe27b80 10 client.0 _ll_register_callbacks cb 0x7f055800ed40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 0x5584229fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 0x5584229fa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43486/0 (socket says 172.21.15.82:43486) 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 -- 172.21.15.82:0/1504339052 learned_addr learned my addr 172.21.15.82:0/1504339052 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056222c700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584227f46b0 0x5584229f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 -- 172.21.15.82:0/1504339052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 msgr2=0x5584229fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 -- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584227f46b0 msgr2=0x5584229f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.282+0000 7f056122a700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584227f46b0 0x5584229f9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056122a700 1 -- 172.21.15.82:0/1504339052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558422700c00 con 0x55842299ea90 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056222c700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584227f46b0 0x5584229f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056122a700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 0x5584229fa380 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f054c005950 tx=0x7f054c01a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e12b051e1a2a6db server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f05457fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f05457fa700 1 -- 172.21.15.82:0/1504339052 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c018030 con 0x55842299ea90 2023-07-14T19:29:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f055409f6d0 con 0x55842299ea90 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f05457fa700 1 -- 172.21.15.82:0/1504339052 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f054c012450 con 0x55842299ea90 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f05457fa700 1 -- 172.21.15.82:0/1504339052 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f054c012610 con 0x55842299ea90 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 10 client.15771 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558422716250 con 0x55842299ea90 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f05457fa700 1 -- 172.21.15.82:0/1504339052 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f055409f6d0 con 0x55842299ea90 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f05457fa700 1 -- 172.21.15.82:0/1504339052 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f054c023880 con 0x55842299ea90 2023-07-14T19:29:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 20 client.15771 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558422a06ea0 0x558422a092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 4 client.15771 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.283+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584228e2e50 con 0x558422a06ea0 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.284+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558422a06ea0 0x558422a092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.284+0000 7f05457fa700 10 client.15771 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.284+0000 7f0561a2b700 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558422a06ea0 0x558422a092b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d7d3046b60a2a9 server_cookie=d4c61d36e23b01a4 in_seq=0 out_seq=0 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f05457fa700 1 -- 172.21.15.82:0/1504339052 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584228e2e50 con 0x558422a06ea0 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f05457fa700 10 client.15771 handle_command_reply: tid=0 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 client.15771 shutdown 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 msgr2=0x5584229fa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 0x5584229fa380 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f054c005950 tx=0x7f054c01a500 comp rx=0 tx=0).stop 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 shutdown_connections 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558422a06ea0 0x558422a092b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55842299ef00 0x5584229fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584227f46b0 0x5584229f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f05457fa700 0 client.15771 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 --2- 172.21.15.82:0/1504339052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55842299ea90 0x5584229fa380 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 >> 172.21.15.82:0/1504339052 conn(0x55842298e3d0 msgr2=0x5584229a0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 shutdown_connections 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 1 -- 172.21.15.82:0/1504339052 wait complete. 2023-07-14T19:29:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.288+0000 7f056fe27b80 20 client.15771 trim_cache size 0 max 16384 2023-07-14T19:29:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:29:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:29:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:29:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:29:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f05480a13e0 msgr2=0x7f05480a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f05480a13e0 0x7f05480a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0558005fd0 tx=0x7f0558005db0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 msgr2=0x7f055c1a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:32.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 0x7f055c1a4050 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f05540077f0 tx=0x7f05540078d0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 shutdown_connections 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f05480a13e0 0x7f05480a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f055c105cd0 0x7f055c1a4050 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f055c068980 0x7f055c0feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 --2- 172.21.15.82:0/1438432958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f055c069330 0x7f055c0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 >> 172.21.15.82:0/1438432958 conn(0x7f055c076ae0 msgr2=0x7f055c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 shutdown_connections 2023-07-14T19:29:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:32.289+0000 7f0563c8f700 1 -- 172.21.15.82:0/1438432958 wait complete. 2023-07-14T19:29:32.382 INFO:tasks.workunit.client.0.smithi082.stderr:Note: switching to '81c9387d2e7576da065747ec9c16d1c7d69016ec'. 2023-07-14T19:29:32.382 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.383 INFO:tasks.workunit.client.0.smithi082.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-14T19:29:32.383 INFO:tasks.workunit.client.0.smithi082.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-14T19:29:32.383 INFO:tasks.workunit.client.0.smithi082.stderr:state without impacting any branches by switching back to a branch. 2023-07-14T19:29:32.383 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.383 INFO:tasks.workunit.client.0.smithi082.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-14T19:29:32.384 INFO:tasks.workunit.client.0.smithi082.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-14T19:29:32.384 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.384 INFO:tasks.workunit.client.0.smithi082.stderr: git switch -c 2023-07-14T19:29:32.384 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.385 INFO:tasks.workunit.client.0.smithi082.stderr:Or undo this operation with: 2023-07-14T19:29:32.385 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.385 INFO:tasks.workunit.client.0.smithi082.stderr: git switch - 2023-07-14T19:29:32.385 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.385 INFO:tasks.workunit.client.0.smithi082.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-14T19:29:32.386 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:32.386 INFO:tasks.workunit.client.0.smithi082.stderr:HEAD is now at 81c9387d2e7 Merge PR #52196 into wip-pdonnell-testing-20230714.134335 2023-07-14T19:29:32.388 DEBUG:teuthology.orchestra.run.smithi082:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-14T19:29:32.414 INFO:tasks.workunit.client.0.smithi082.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-14T19:29:32.418 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-14T19:29:32.418 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-14T19:29:32.977 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-14T19:29:33.039 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-14T19:29:33.076 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-14T19:29:33.077 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-14T19:29:33.078 INFO:tasks.workunit.client.0.smithi082.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-14T19:29:33.114 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-14T19:29:33.117 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:29:33.118 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-14T19:29:33.175 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-07-14T19:29:33.177 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-07-14T19:29:33.178 DEBUG:teuthology.orchestra.run.smithi082:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81c9387d2e7576da065747ec9c16d1c7d69016ec TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-07-14T19:29:33.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:32 smithi133 ceph-mon[128894]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-14T19:29:33.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:32 smithi082 ceph-mon[127461]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-14T19:29:33.239 INFO:tasks.workunit.client.0.smithi082.stdout:getting blogbench 2023-07-14T19:29:33.240 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'getting blogbench' 2023-07-14T19:29:33.240 INFO:tasks.workunit.client.0.smithi082.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-07-14T19:29:33.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:32 smithi161 ceph-mon[129184]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-14T19:29:33.261 INFO:tasks.workunit.client.0.smithi082.stderr:--2023-07-14 19:29:33-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-07-14T19:29:33.262 INFO:tasks.workunit.client.0.smithi082.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-07-14T19:29:33.311 INFO:tasks.workunit.client.0.smithi082.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-07-14T19:29:33.362 INFO:tasks.workunit.client.0.smithi082.stderr:HTTP request sent, awaiting response... 200 OK 2023-07-14T19:29:33.362 INFO:tasks.workunit.client.0.smithi082.stderr:Length: 103355 (101K) [application/octet-stream] 2023-07-14T19:29:33.363 INFO:tasks.workunit.client.0.smithi082.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-07-14T19:29:33.366 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:33.470 INFO:tasks.workunit.client.0.smithi082.stderr: 0K .......... .......... .......... .......... .......... 49% 475K 0s 2023-07-14T19:29:33.518 INFO:tasks.workunit.client.0.smithi082.stderr: 50K .......... .......... .......... .......... .......... 99% 1009K 0s 2023-07-14T19:29:33.518 INFO:tasks.workunit.client.0.smithi082.stderr: 100K 100% 1779G=0.2s 2023-07-14T19:29:33.518 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:33.519 INFO:tasks.workunit.client.0.smithi082.stderr:2023-07-14 19:29:33 (652 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-07-14T19:29:33.519 INFO:tasks.workunit.client.0.smithi082.stderr: 2023-07-14T19:29:33.519 INFO:tasks.workunit.client.0.smithi082.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-07-14T19:29:33.544 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/ 2023-07-14T19:29:33.545 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/m4/ 2023-07-14T19:29:33.546 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-07-14T19:29:33.547 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/ 2023-07-14T19:29:33.548 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/blogbench.8 2023-07-14T19:29:33.549 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/Makefile.am 2023-07-14T19:29:33.550 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/man/Makefile.in 2023-07-14T19:29:33.550 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/ 2023-07-14T19:29:33.551 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/mysnprintf.c 2023-07-14T19:29:33.552 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/mysnprintf.h 2023-07-14T19:29:33.553 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/blogbench.c 2023-07-14T19:29:33.554 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/blogbench.h 2023-07-14T19:29:33.555 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/fakesnprintf.c 2023-07-14T19:29:33.556 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/fakesnprintf.h 2023-07-14T19:29:33.557 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/blogbench_p.h 2023-07-14T19:29:33.558 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-07-14T19:29:33.558 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-07-14T19:29:33.559 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/Makefile.am 2023-07-14T19:29:33.560 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/Makefile.in 2023-07-14T19:29:33.561 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/process.c 2023-07-14T19:29:33.562 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/globals.h 2023-07-14T19:29:33.563 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/commenter.c 2023-07-14T19:29:33.568 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/rewriter.c 2023-07-14T19:29:33.569 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/reader.c 2023-07-14T19:29:33.570 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/writer.c 2023-07-14T19:29:33.570 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/src/helpers.c 2023-07-14T19:29:33.571 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/NEWS 2023-07-14T19:29:33.572 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/compile 2023-07-14T19:29:33.573 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/depcomp 2023-07-14T19:29:33.574 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/aclocal.m4 2023-07-14T19:29:33.575 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/README 2023-07-14T19:29:33.576 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/configure 2023-07-14T19:29:33.579 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/configure.ac 2023-07-14T19:29:33.580 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/config.guess 2023-07-14T19:29:33.581 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/install-sh 2023-07-14T19:29:33.582 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/config.sub 2023-07-14T19:29:33.583 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/missing 2023-07-14T19:29:33.583 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/mkinstalldirs 2023-07-14T19:29:33.584 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/Makefile.am 2023-07-14T19:29:33.585 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/Makefile.in 2023-07-14T19:29:33.585 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/config.h.in 2023-07-14T19:29:33.586 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/AUTHORS 2023-07-14T19:29:33.587 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/INSTALL 2023-07-14T19:29:33.588 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/ChangeLog 2023-07-14T19:29:33.588 INFO:tasks.workunit.client.0.smithi082.stdout:blogbench-1.0/COPYING 2023-07-14T19:29:33.589 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd blogbench-1.0/ 2023-07-14T19:29:33.590 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'making blogbench' 2023-07-14T19:29:33.590 INFO:tasks.workunit.client.0.smithi082.stdout:making blogbench 2023-07-14T19:29:33.591 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./configure 2023-07-14T19:29:33.696 INFO:tasks.workunit.client.0.smithi082.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-14T19:29:34.703 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether build environment is sane... yes 2023-07-14T19:29:34.712 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gawk... gawk 2023-07-14T19:29:34.744 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether make sets $(MAKE)... yes 2023-07-14T19:29:34.759 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc... gcc 2023-07-14T19:29:34.811 INFO:tasks.workunit.client.0.smithi082.stdout:checking for C compiler default output file name... a.out 2023-07-14T19:29:34.813 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether the C compiler works... yes 2023-07-14T19:29:34.815 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are cross compiling... no 2023-07-14T19:29:34.851 INFO:tasks.workunit.client.0.smithi082.stdout:checking for suffix of executables... 2023-07-14T19:29:34.885 INFO:tasks.workunit.client.0.smithi082.stdout:checking for suffix of object files... o 2023-07-14T19:29:34.916 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether we are using the GNU C compiler... yes 2023-07-14T19:29:34.948 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc accepts -g... yes 2023-07-14T19:29:34.989 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gcc option to accept ANSI C... none needed 2023-07-14T19:29:35.029 INFO:tasks.workunit.client.0.smithi082.stdout:checking for style of include used by make... GNU 2023-07-14T19:29:35.113 INFO:tasks.workunit.client.0.smithi082.stdout:checking dependency style of gcc... gcc3 2023-07-14T19:29:35.113 INFO:tasks.workunit.client.0.smithi082.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-07-14T19:29:35.164 INFO:tasks.workunit.client.0.smithi082.stdout:checking how to run the C preprocessor... gcc -E 2023-07-14T19:29:35.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:34 smithi133 ceph-mon[128894]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T19:29:35.199 INFO:tasks.workunit.client.0.smithi082.stdout:checking for egrep... grep -E 2023-07-14T19:29:35.206 INFO:tasks.workunit.client.0.smithi082.stdout:checking for AIX... no 2023-07-14T19:29:35.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:34 smithi082 ceph-mon[127461]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T19:29:35.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:34 smithi161 ceph-mon[129184]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-07-14T19:29:35.264 INFO:tasks.workunit.client.0.smithi082.stdout:checking for library containing strerror... none required 2023-07-14T19:29:35.387 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ANSI C header files... yes 2023-07-14T19:29:35.423 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-07-14T19:29:35.478 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/types.h... yes 2023-07-14T19:29:35.615 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/stat.h... yes 2023-07-14T19:29:35.660 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdlib.h... yes 2023-07-14T19:29:35.708 INFO:tasks.workunit.client.0.smithi082.stdout:checking for string.h... yes 2023-07-14T19:29:35.757 INFO:tasks.workunit.client.0.smithi082.stdout:checking for memory.h... yes 2023-07-14T19:29:35.803 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strings.h... yes 2023-07-14T19:29:35.850 INFO:tasks.workunit.client.0.smithi082.stdout:checking for inttypes.h... yes 2023-07-14T19:29:35.895 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdint.h... yes 2023-07-14T19:29:35.943 INFO:tasks.workunit.client.0.smithi082.stdout:checking for unistd.h... yes 2023-07-14T19:29:35.954 INFO:tasks.workunit.client.0.smithi082.stdout:checking for memory.h... (cached) yes 2023-07-14T19:29:35.959 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strings.h... (cached) yes 2023-07-14T19:29:35.963 INFO:tasks.workunit.client.0.smithi082.stdout:checking for unistd.h... (cached) yes 2023-07-14T19:29:36.005 INFO:tasks.workunit.client.0.smithi082.stdout:checking locale.h usability... yes 2023-07-14T19:29:36.022 INFO:tasks.workunit.client.0.smithi082.stdout:checking locale.h presence... yes 2023-07-14T19:29:36.023 INFO:tasks.workunit.client.0.smithi082.stdout:checking for locale.h... yes 2023-07-14T19:29:36.037 INFO:tasks.workunit.client.0.smithi082.stdout:checking for stdlib.h... (cached) yes 2023-07-14T19:29:36.041 INFO:tasks.workunit.client.0.smithi082.stdout:checking for string.h... (cached) yes 2023-07-14T19:29:36.085 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/socket.h usability... yes 2023-07-14T19:29:36.103 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/socket.h presence... yes 2023-07-14T19:29:36.103 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/socket.h... yes 2023-07-14T19:29:36.151 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/time.h usability... yes 2023-07-14T19:29:36.168 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/time.h presence... yes 2023-07-14T19:29:36.169 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/time.h... yes 2023-07-14T19:29:36.219 INFO:tasks.workunit.client.0.smithi082.stdout:checking fcntl.h usability... yes 2023-07-14T19:29:36.236 INFO:tasks.workunit.client.0.smithi082.stdout:checking fcntl.h presence... yes 2023-07-14T19:29:36.237 INFO:tasks.workunit.client.0.smithi082.stdout:checking for fcntl.h... yes 2023-07-14T19:29:36.286 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fcntl.h usability... yes 2023-07-14T19:29:36.302 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/fcntl.h presence... yes 2023-07-14T19:29:36.303 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/fcntl.h... yes 2023-07-14T19:29:36.360 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/param.h usability... yes 2023-07-14T19:29:36.379 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/param.h presence... yes 2023-07-14T19:29:36.380 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/param.h... yes 2023-07-14T19:29:36.420 INFO:tasks.workunit.client.0.smithi082.stdout:checking ioctl.h usability... no 2023-07-14T19:29:36.436 INFO:tasks.workunit.client.0.smithi082.stdout:checking ioctl.h presence... no 2023-07-14T19:29:36.436 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ioctl.h... no 2023-07-14T19:29:36.482 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/ioctl.h usability... yes 2023-07-14T19:29:36.498 INFO:tasks.workunit.client.0.smithi082.stdout:checking sys/ioctl.h presence... yes 2023-07-14T19:29:36.499 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sys/ioctl.h... yes 2023-07-14T19:29:36.546 INFO:tasks.workunit.client.0.smithi082.stdout:checking return type of signal handlers... void 2023-07-14T19:29:36.588 INFO:tasks.workunit.client.0.smithi082.stdout:checking for size_t... yes 2023-07-14T19:29:36.624 INFO:tasks.workunit.client.0.smithi082.stdout:checking for dev_t... yes 2023-07-14T19:29:36.664 INFO:tasks.workunit.client.0.smithi082.stdout:checking for short... yes 2023-07-14T19:29:36.732 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of short... 2 2023-07-14T19:29:36.777 INFO:tasks.workunit.client.0.smithi082.stdout:checking for int... yes 2023-07-14T19:29:36.841 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of int... 4 2023-07-14T19:29:36.888 INFO:tasks.workunit.client.0.smithi082.stdout:checking for long... yes 2023-07-14T19:29:36.954 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of long... 8 2023-07-14T19:29:36.996 INFO:tasks.workunit.client.0.smithi082.stdout:checking for long long... yes 2023-07-14T19:29:37.059 INFO:tasks.workunit.client.0.smithi082.stdout:checking size of long long... 8 2023-07-14T19:29:37.098 INFO:tasks.workunit.client.0.smithi082.stdout:checking for ssize_t... yes 2023-07-14T19:29:37.135 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sig_atomic_t... yes 2023-07-14T19:29:37.174 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether gcc needs -traditional... no 2023-07-14T19:29:37.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:36 smithi133 ceph-mon[128894]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-07-14T19:29:37.206 INFO:tasks.workunit.client.0.smithi082.stdout:checking for an ANSI C-conforming const... yes 2023-07-14T19:29:37.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:36 smithi082 ceph-mon[127461]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-07-14T19:29:37.228 INFO:tasks.workunit.client.0.smithi082.stdout:checking for inline... inline 2023-07-14T19:29:37.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:36 smithi161 ceph-mon[129184]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-07-14T19:29:37.282 INFO:tasks.workunit.client.0.smithi082.stdout:checking for working alloca.h... yes 2023-07-14T19:29:37.325 INFO:tasks.workunit.client.0.smithi082.stdout:checking for alloca... yes 2023-07-14T19:29:37.380 INFO:tasks.workunit.client.0.smithi082.stdout:checking for working memcmp... yes 2023-07-14T19:29:37.431 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strftime... yes 2023-07-14T19:29:37.498 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-07-14T19:29:37.553 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether stat accepts an empty string... no 2023-07-14T19:29:37.604 INFO:tasks.workunit.client.0.smithi082.stdout:checking for vprintf... yes 2023-07-14T19:29:37.656 INFO:tasks.workunit.client.0.smithi082.stdout:checking for _doprnt... no 2023-07-14T19:29:37.679 INFO:tasks.workunit.client.0.smithi082.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-07-14T19:29:37.691 INFO:tasks.workunit.client.0.smithi082.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-07-14T19:29:37.765 INFO:tasks.workunit.client.0.smithi082.stdout:checking for the pthreads library -lpthreads... no 2023-07-14T19:29:37.813 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether pthreads work without any flags... no 2023-07-14T19:29:37.828 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether pthreads work with -Kthread... no 2023-07-14T19:29:37.843 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether pthreads work with -kthread... no 2023-07-14T19:29:37.884 INFO:tasks.workunit.client.0.smithi082.stdout:checking for the pthreads library -llthread... no 2023-07-14T19:29:37.932 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether pthreads work with -pthread... yes 2023-07-14T19:29:37.976 INFO:tasks.workunit.client.0.smithi082.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-07-14T19:29:37.977 INFO:tasks.workunit.client.0.smithi082.stdout:checking if more special flags are required for pthreads... no 2023-07-14T19:29:37.977 INFO:tasks.workunit.client.0.smithi082.stdout:checking for cc_r... gcc 2023-07-14T19:29:38.053 INFO:tasks.workunit.client.0.smithi082.stdout:checking for setrlimit... yes 2023-07-14T19:29:38.109 INFO:tasks.workunit.client.0.smithi082.stdout:checking for setproctitle... no 2023-07-14T19:29:38.163 INFO:tasks.workunit.client.0.smithi082.stdout:checking for getopt_long... yes 2023-07-14T19:29:38.219 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strtoull... yes 2023-07-14T19:29:38.276 INFO:tasks.workunit.client.0.smithi082.stdout:checking for strtoq... yes 2023-07-14T19:29:38.332 INFO:tasks.workunit.client.0.smithi082.stdout:checking for snprintf... yes 2023-07-14T19:29:38.387 INFO:tasks.workunit.client.0.smithi082.stdout:checking for vsnprintf... yes 2023-07-14T19:29:38.444 INFO:tasks.workunit.client.0.smithi082.stdout:checking for vfprintf... yes 2023-07-14T19:29:38.500 INFO:tasks.workunit.client.0.smithi082.stdout:checking for gethostname... yes 2023-07-14T19:29:38.556 INFO:tasks.workunit.client.0.smithi082.stdout:checking for setlocale... yes 2023-07-14T19:29:38.612 INFO:tasks.workunit.client.0.smithi082.stdout:checking for sync... yes 2023-07-14T19:29:38.672 INFO:tasks.workunit.client.0.smithi082.stdout:checking whether snprintf is C99 conformant... done 2023-07-14T19:29:38.683 INFO:tasks.workunit.client.0.smithi082.stdout:configure: creating ./config.status 2023-07-14T19:29:38.999 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating Makefile 2023-07-14T19:29:39.016 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating src/Makefile 2023-07-14T19:29:39.035 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating man/Makefile 2023-07-14T19:29:39.046 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: creating config.h 2023-07-14T19:29:39.094 INFO:tasks.workunit.client.0.smithi082.stdout:config.status: executing depfiles commands 2023-07-14T19:29:39.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:38 smithi133 ceph-mon[128894]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 5 op/s 2023-07-14T19:29:39.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:38 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:29:39.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:38 smithi082 ceph-mon[127461]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 5 op/s 2023-07-14T19:29:39.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:38 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:29:39.222 INFO:tasks.workunit.client.0.smithi082.stderr:+ make 2023-07-14T19:29:39.234 INFO:tasks.workunit.client.0.smithi082.stdout:make all-recursive 2023-07-14T19:29:39.235 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-14T19:29:39.237 INFO:tasks.workunit.client.0.smithi082.stdout:Making all in src 2023-07-14T19:29:39.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:38 smithi161 ceph-mon[129184]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 5 op/s 2023-07-14T19:29:39.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:38 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:29:39.252 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-07-14T19:29:39.253 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-07-14T19:29:39.253 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-07-14T19:29:39.340 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-07-14T19:29:39.341 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-07-14T19:29:39.402 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-07-14T19:29:39.403 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-07-14T19:29:39.453 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-07-14T19:29:39.454 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-07-14T19:29:39.501 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-07-14T19:29:39.501 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-07-14T19:29:39.550 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-07-14T19:29:39.551 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-07-14T19:29:39.595 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-07-14T19:29:39.595 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-07-14T19:29:39.662 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-07-14T19:29:39.662 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-07-14T19:29:39.693 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-07-14T19:29:39.693 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-07-14T19:29:39.725 INFO:tasks.workunit.client.0.smithi082.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-07-14T19:29:39.725 INFO:tasks.workunit.client.0.smithi082.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-07-14T19:29:39.760 INFO:tasks.workunit.client.0.smithi082.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-07-14T19:29:39.778 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-07-14T19:29:39.778 INFO:tasks.workunit.client.0.smithi082.stdout:Making all in man 2023-07-14T19:29:39.796 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-07-14T19:29:39.796 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Nothing to be done for 'all'. 2023-07-14T19:29:39.796 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-07-14T19:29:39.797 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-14T19:29:39.798 INFO:tasks.workunit.client.0.smithi082.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-14T19:29:39.798 INFO:tasks.workunit.client.0.smithi082.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-07-14T19:29:39.799 INFO:tasks.workunit.client.0.smithi082.stderr:+ cd src 2023-07-14T19:29:39.799 INFO:tasks.workunit.client.0.smithi082.stderr:+ mkdir blogtest_in 2023-07-14T19:29:39.800 INFO:tasks.workunit.client.0.smithi082.stdout:running blogbench 2023-07-14T19:29:39.801 INFO:tasks.workunit.client.0.smithi082.stderr:+ echo 'running blogbench' 2023-07-14T19:29:39.802 INFO:tasks.workunit.client.0.smithi082.stderr:+ ./blogbench -d blogtest_in 2023-07-14T19:29:39.803 INFO:tasks.workunit.client.0.smithi082.stdout: 2023-07-14T19:29:39.803 INFO:tasks.workunit.client.0.smithi082.stdout:Frequency = 10 secs 2023-07-14T19:29:39.804 INFO:tasks.workunit.client.0.smithi082.stdout:Scratch dir = [blogtest_in] 2023-07-14T19:29:40.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:29:40.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:29:40.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:29:40.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:29:40.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:29:40.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:29:40.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:29:40.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:29:40.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:29:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:40 smithi133 ceph-mon[128894]: pgmap v37: 97 pgs: 97 active+clean; 5.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-07-14T19:29:41.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:40 smithi082 ceph-mon[127461]: pgmap v37: 97 pgs: 97 active+clean; 5.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-07-14T19:29:41.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:40 smithi161 ceph-mon[129184]: pgmap v37: 97 pgs: 97 active+clean; 5.2 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 344 KiB/s wr, 29 op/s 2023-07-14T19:29:42.168 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 3 writers... 2023-07-14T19:29:42.168 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 1 rewriters... 2023-07-14T19:29:42.168 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 5 commenters... 2023-07-14T19:29:42.168 INFO:tasks.workunit.client.0.smithi082.stdout:Spawning 100 readers... 2023-07-14T19:29:42.170 INFO:tasks.workunit.client.0.smithi082.stdout:Benchmarking for 30 iterations. 2023-07-14T19:29:42.170 INFO:tasks.workunit.client.0.smithi082.stdout:The test will run during 5 minutes. 2023-07-14T19:29:42.170 INFO:tasks.workunit.client.0.smithi082.stdout: 2023-07-14T19:29:42.170 INFO:tasks.workunit.client.0.smithi082.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-07-14T19:29:43.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:42 smithi133 ceph-mon[128894]: pgmap v38: 97 pgs: 97 active+clean; 5.7 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 396 KiB/s wr, 36 op/s 2023-07-14T19:29:43.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:42 smithi082 ceph-mon[127461]: pgmap v38: 97 pgs: 97 active+clean; 5.7 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 396 KiB/s wr, 36 op/s 2023-07-14T19:29:43.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:42 smithi161 ceph-mon[129184]: pgmap v38: 97 pgs: 97 active+clean; 5.7 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 396 KiB/s wr, 36 op/s 2023-07-14T19:29:45.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:44 smithi133 ceph-mon[128894]: pgmap v39: 97 pgs: 97 active+clean; 5.7 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 396 KiB/s wr, 35 op/s 2023-07-14T19:29:45.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:44 smithi082 ceph-mon[127461]: pgmap v39: 97 pgs: 97 active+clean; 5.7 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 396 KiB/s wr, 35 op/s 2023-07-14T19:29:45.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:44 smithi161 ceph-mon[129184]: pgmap v39: 97 pgs: 97 active+clean; 5.7 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 396 KiB/s wr, 35 op/s 2023-07-14T19:29:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 -- 172.21.15.82:0/3097303491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 msgr2=0x7efdbc114270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 --2- 172.21.15.82:0/3097303491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc114270 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7efdb400b0d0 tx=0x7efdb400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 -- 172.21.15.82:0/3097303491 shutdown_connections 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 --2- 172.21.15.82:0/3097303491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc114270 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 --2- 172.21.15.82:0/3097303491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdbc1078b0 0x7efdbc107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 --2- 172.21.15.82:0/3097303491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdbc108250 0x7efdbc10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 -- 172.21.15.82:0/3097303491 >> 172.21.15.82:0/3097303491 conn(0x7efdbc071260 msgr2=0x7efdbc06df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 -- 172.21.15.82:0/3097303491 shutdown_connections 2023-07-14T19:29:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.407+0000 7efdc2790700 1 -- 172.21.15.82:0/3097303491 wait complete. 2023-07-14T19:29:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 Processor -- start 2023-07-14T19:29:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 -- start start 2023-07-14T19:29:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdbc1078b0 0x7efdbc1a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdbc108250 0x7efdbc1a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc19da90 con 0x7efdbc10c6c0 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc19dc00 con 0x7efdbc108250 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc2790700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc19ddc0 con 0x7efdbc1078b0 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdbc108250 0x7efdbc1a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43528/0 (socket says 172.21.15.82:43528) 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3643856654 learned_addr learned my addr 172.21.15.82:0/3643856654 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdbbfff700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdbc1078b0 0x7efdbc1a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3643856654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdbc1078b0 msgr2=0x7efdbc1a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdbc1078b0 0x7efdbc1a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3643856654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdbc108250 msgr2=0x7efdbc1a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdbc108250 0x7efdbc1a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.408+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3643856654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb4009d00 con 0x7efdbc10c6c0 2023-07-14T19:29:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.409+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc19d430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007870 tx=0x7efdb40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.409+0000 7efdb9ffb700 1 -- 172.21.15.82:0/3643856654 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb400e040 con 0x7efdbc10c6c0 2023-07-14T19:29:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.409+0000 7efdb9ffb700 1 -- 172.21.15.82:0/3643856654 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb4005070 con 0x7efdbc10c6c0 2023-07-14T19:29:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.409+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdbc19e040 con 0x7efdbc10c6c0 2023-07-14T19:29:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.409+0000 7efdb9ffb700 1 -- 172.21.15.82:0/3643856654 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb401a700 con 0x7efdbc10c6c0 2023-07-14T19:29:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.409+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc068bc0 con 0x7efdbc10c6c0 2023-07-14T19:29:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.411+0000 7efdb9ffb700 1 -- 172.21.15.82:0/3643856654 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7efdb4020070 con 0x7efdbc10c6c0 2023-07-14T19:29:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.411+0000 7efdb9ffb700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efda80a13e0 0x7efda80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.411+0000 7efdbbfff700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efda80a13e0 0x7efda80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.411+0000 7efdb9ffb700 1 -- 172.21.15.82:0/3643856654 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7efdb40cf9b0 con 0x7efdbc10c6c0 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.412+0000 7efdbbfff700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efda80a13e0 0x7efda80a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efdbc108670 tx=0x7efdac009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.412+0000 7efdce928b80 1 Processor -- start 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.412+0000 7efdce928b80 1 -- start start 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.412+0000 7efdce928b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a9606fd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdce928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a9606f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdce928b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a9606f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdce928b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9605fcc50 con 0x55a9606fdf00 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdce928b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a96064f9e0 con 0x55a9605536b0 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdce928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a960648e70 con 0x55a9606fda90 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a9606f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a9606f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43536/0 (socket says 172.21.15.82:43536) 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 -- 172.21.15.82:0/1054070128 learned_addr learned my addr 172.21.15.82:0/1054070128 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdbbfff700 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a9606fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdb3fff700 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a9606f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 -- 172.21.15.82:0/1054070128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 msgr2=0x55a9606f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a9606f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 -- 172.21.15.82:0/1054070128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 msgr2=0x55a9606fd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a9606fd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 -- 172.21.15.82:0/1054070128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a960476d80 con 0x55a9606fdf00 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a9606f7f80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efdb409c760 tx=0x7efdb4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9a4372e11ea83 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdb27fc700 1 -- 172.21.15.82:0/1054070128 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40054e0 con 0x55a9606fdf00 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdb27fc700 1 -- 172.21.15.82:0/1054070128 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb409ca60 con 0x55a9606fdf00 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.413+0000 7efdb27fc700 1 -- 172.21.15.82:0/1054070128 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb409c170 con 0x55a9606fdf00 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 -- 172.21.15.82:0/1054070128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 msgr2=0x55a9606f7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a9606f7f80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efdb409c760 tx=0x7efdb4009b80 comp rx=0 tx=0).stop 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 -- 172.21.15.82:0/1054070128 shutdown_connections 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a9606f7f80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a9606fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 --2- 172.21.15.82:0/1054070128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a9606f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 -- 172.21.15.82:0/1054070128 >> 172.21.15.82:0/1054070128 conn(0x55a9606ed3d0 msgr2=0x55a9606f2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 -- 172.21.15.82:0/1054070128 shutdown_connections 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 -- 172.21.15.82:0/1054070128 wait complete. 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.414+0000 7efdce928b80 1 -- start start 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 Processor -- start 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9605536b0 0x55a960706bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9607070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9606fdf00 0x55a960700eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9605e8a60 con 0x55a9606fda90 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9605e8760 con 0x55a9606fdf00 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9604cc8c0 con 0x55a9605536b0 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdce928b80 10 client.0 _ll_register_callbacks cb 0x55a9605e6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9607070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9607070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43540/0 (socket says 172.21.15.82:43540) 2023-07-14T19:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 -- 172.21.15.82:0/2733342944 learned_addr learned my addr 172.21.15.82:0/2733342944 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdbbfff700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9605536b0 0x55a960706bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9606fdf00 0x55a960700eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 -- 172.21.15.82:0/2733342944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9605536b0 msgr2=0x55a960706bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9605536b0 0x55a960706bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 -- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9606fdf00 msgr2=0x55a960700eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9606fdf00 0x55a960700eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.415+0000 7efdb3fff700 1 -- 172.21.15.82:0/2733342944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdbc068bc0 con 0x55a9606fda90 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdb3fff700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9607070f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efda400b9c0 tx=0x7efda400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf993ac8cb62aad3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efd9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efd9b7fe700 1 -- 172.21.15.82:0/2733342944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4003d70 con 0x55a9606fda90 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efd9b7fe700 1 -- 172.21.15.82:0/2733342944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7efda4004830 con 0x55a9606fda90 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9605a2490 con 0x55a9606fda90 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efd9b7fe700 1 -- 172.21.15.82:0/2733342944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efda40049f0 con 0x55a9606fda90 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 10 client.15783 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a96045fc00 con 0x55a9606fda90 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efd9b7fe700 1 -- 172.21.15.82:0/2733342944 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9605a2490 con 0x55a9606fda90 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efd9b7fe700 1 -- 172.21.15.82:0/2733342944 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efda4010440 con 0x55a9606fda90 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 10 client.15783 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 20 client.15783 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960711140 0x55a960713550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 4 client.15783 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.416+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9606c6d40 con 0x55a960711140 2023-07-14T19:29:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.417+0000 7efdbbfff700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960711140 0x55a960713550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.417+0000 7efd9b7fe700 10 client.15783 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.417+0000 7efdbbfff700 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960711140 0x55a960713550 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c240542191a8cded server_cookie=6558a0e025f2d0f3 in_seq=0 out_seq=0 2023-07-14T19:29:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.417+0000 7efd9b7fe700 1 -- 172.21.15.82:0/2733342944 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a9606c6d40 con 0x55a960711140 2023-07-14T19:29:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.417+0000 7efd9b7fe700 10 client.15783 handle_command_reply: tid=0 2023-07-14T19:29:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.417+0000 7efdce928b80 1 client.15783 shutdown 2023-07-14T19:29:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 msgr2=0x55a9607070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9607070f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efda400b9c0 tx=0x7efda400bcd0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 shutdown_connections 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9606fdf00 0x55a960700eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960711140 0x55a960713550 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9605536b0 0x55a960706bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 --2- 172.21.15.82:0/2733342944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9607070f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efd9b7fe700 0 client.15783 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 >> 172.21.15.82:0/2733342944 conn(0x55a9606ed3d0 msgr2=0x55a9606f1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 shutdown_connections 2023-07-14T19:29:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 1 -- 172.21.15.82:0/2733342944 wait complete. 2023-07-14T19:29:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.418+0000 7efdce928b80 20 client.15783 trim_cache size 0 max 16384 2023-07-14T19:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 Processor -- start 2023-07-14T19:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 -- start start 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960700fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9606fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fdf00 0x55a9606ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a96070e730 con 0x55a9606fda90 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9606ff430 con 0x55a9605536b0 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdce928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9606f4480 con 0x55a9606fdf00 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9606fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdb3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9606fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43548/0 (socket says 172.21.15.82:43548) 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdb3fff700 1 -- 172.21.15.82:0/2541368664 learned_addr learned my addr 172.21.15.82:0/2541368664 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdbbfff700 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960700fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdb3fff700 1 -- 172.21.15.82:0/2541368664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fdf00 msgr2=0x55a9606ffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.433+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fdf00 0x55a9606ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb3fff700 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fdf00 0x55a9606ffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb3fff700 1 -- 172.21.15.82:0/2541368664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 msgr2=0x55a960700fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb3fff700 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960700fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb3fff700 1 -- 172.21.15.82:0/2541368664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a96049d1f0 con 0x55a9606fda90 2023-07-14T19:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fdf00 0x55a9606ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb3fff700 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9606fe980 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efda4010850 tx=0x7efda4003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28a49a93fae805c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb2ffd700 1 -- 172.21.15.82:0/2541368664 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4010440 con 0x55a9606fda90 2023-07-14T19:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb2ffd700 1 -- 172.21.15.82:0/2541368664 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda40105a0 con 0x55a9606fda90 2023-07-14T19:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdb2ffd700 1 -- 172.21.15.82:0/2541368664 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4011a20 con 0x55a9606fda90 2023-07-14T19:29:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdce928b80 1 -- 172.21.15.82:0/2541368664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 msgr2=0x55a9606fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdce928b80 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9606fe980 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efda4010850 tx=0x7efda4003a70 comp rx=0 tx=0).stop 2023-07-14T19:29:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdce928b80 1 -- 172.21.15.82:0/2541368664 shutdown_connections 2023-07-14T19:29:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.434+0000 7efdce928b80 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fdf00 0x55a9606ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960700fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 --2- 172.21.15.82:0/2541368664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fda90 0x55a9606fe980 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- 172.21.15.82:0/2541368664 >> 172.21.15.82:0/2541368664 conn(0x55a9606ed3d0 msgr2=0x55a9606ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- 172.21.15.82:0/2541368664 shutdown_connections 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- 172.21.15.82:0/2541368664 wait complete. 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- start start 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 Processor -- start 2023-07-14T19:29:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960706f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a960705230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a960705790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a960705ea0 con 0x55a9606fdf00 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a960706010 con 0x55a9605536b0 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a960764df0 con 0x55a9606fda90 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.435+0000 7efdce928b80 10 client.0 _ll_register_callbacks cb 0x7efda4004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a960705790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdbbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960706f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a960705790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43556/0 (socket says 172.21.15.82:43556) 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3435129088 learned_addr learned my addr 172.21.15.82:0/3435129088 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdb3fff700 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a960705230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3435129088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 msgr2=0x55a960705230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a960705230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 msgr2=0x55a960706f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960706f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 -- 172.21.15.82:0/3435129088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a96045fc00 con 0x55a9606fdf00 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdc0d2d700 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a960705790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005890 tx=0x7efdb409c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8026cdb3e67c9f61 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efd9b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efd9b7fe700 1 -- 172.21.15.82:0/3435129088 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40054e0 con 0x55a9606fdf00 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efd9b7fe700 1 -- 172.21.15.82:0/3435129088 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7efdb40a4e20 con 0x55a9606fdf00 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdb409ca60 con 0x55a9606fdf00 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efd9b7fe700 1 -- 172.21.15.82:0/3435129088 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdb40a0a90 con 0x55a9606fdf00 2023-07-14T19:29:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdce928b80 10 client.15789 fetch_fsmap learned FSMap version 28 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a960475250 con 0x55a9606fdf00 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.436+0000 7efd9b7fe700 1 -- 172.21.15.82:0/3435129088 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdb407f020 con 0x55a9606fdf00 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efd9b7fe700 1 -- 172.21.15.82:0/3435129088 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdb40cd2f0 con 0x55a9606fdf00 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdce928b80 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdce928b80 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdce928b80 20 client.15789 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdce928b80 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960766020 0x55a960768430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdce928b80 4 client.15789 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:29:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a960641e50 con 0x55a960766020 2023-07-14T19:29:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdbbfff700 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960766020 0x55a960768430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efd9b7fe700 10 client.15789 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.437+0000 7efdbbfff700 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960766020 0x55a960768430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec76bdad9d666157 server_cookie=f2b5f43244923932 in_seq=0 out_seq=0 2023-07-14T19:29:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efd9b7fe700 1 -- 172.21.15.82:0/3435129088 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a960641e50 con 0x55a960766020 2023-07-14T19:29:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efd9b7fe700 10 client.15789 handle_command_reply: tid=0 2023-07-14T19:29:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 client.15789 shutdown 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 msgr2=0x55a960705790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a960705790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005890 tx=0x7efdb409c180 comp rx=0 tx=0).stop 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 shutdown_connections 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9606fdf00 0x55a960705790 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9605536b0 0x55a960706f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a960766020 0x55a960768430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 --2- 172.21.15.82:0/3435129088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a9606fda90 0x55a960705230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efd9b7fe700 0 client.15789 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 >> 172.21.15.82:0/3435129088 conn(0x55a9606ed3d0 msgr2=0x55a960700e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 shutdown_connections 2023-07-14T19:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 1 -- 172.21.15.82:0/3435129088 wait complete. 2023-07-14T19:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdce928b80 20 client.15789 trim_cache size 0 max 16384 2023-07-14T19:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efda80a13e0 msgr2=0x7efda80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdc2790700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efda80a13e0 0x7efda80a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efdbc108670 tx=0x7efdac009260 comp rx=0 tx=0).stop 2023-07-14T19:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 msgr2=0x7efdbc19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.444+0000 7efdc2790700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc19d430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7efdb4007870 tx=0x7efdb40078a0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 shutdown_connections 2023-07-14T19:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efda80a13e0 0x7efda80a3890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdbc10c6c0 0x7efdbc19d430 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efdbc1078b0 0x7efdbc1a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 --2- 172.21.15.82:0/3643856654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdbc108250 0x7efdbc1a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 >> 172.21.15.82:0/3643856654 conn(0x7efdbc071260 msgr2=0x7efdbc06c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 shutdown_connections 2023-07-14T19:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:45.445+0000 7efdc2790700 1 -- 172.21.15.82:0/3643856654 wait complete. 2023-07-14T19:29:47.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:46 smithi133 ceph-mon[128894]: pgmap v40: 97 pgs: 97 active+clean; 10 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 888 KiB/s wr, 69 op/s 2023-07-14T19:29:47.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:46 smithi082 ceph-mon[127461]: pgmap v40: 97 pgs: 97 active+clean; 10 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 888 KiB/s wr, 69 op/s 2023-07-14T19:29:47.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:46 smithi161 ceph-mon[129184]: pgmap v40: 97 pgs: 97 active+clean; 10 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 888 KiB/s wr, 69 op/s 2023-07-14T19:29:49.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:48 smithi133 ceph-mon[128894]: pgmap v41: 97 pgs: 97 active+clean; 10 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 877 KiB/s wr, 66 op/s 2023-07-14T19:29:49.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:48 smithi082 ceph-mon[127461]: pgmap v41: 97 pgs: 97 active+clean; 10 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 877 KiB/s wr, 66 op/s 2023-07-14T19:29:49.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:48 smithi161 ceph-mon[129184]: pgmap v41: 97 pgs: 97 active+clean; 10 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 877 KiB/s wr, 66 op/s 2023-07-14T19:29:51.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:50 smithi133 ceph-mon[128894]: pgmap v42: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 241 op/s 2023-07-14T19:29:51.221 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:50 smithi082 ceph-mon[127461]: pgmap v42: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 241 op/s 2023-07-14T19:29:51.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:50 smithi161 ceph-mon[129184]: pgmap v42: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 241 op/s 2023-07-14T19:29:52.170 INFO:tasks.workunit.client.0.smithi082.stdout: 12 291784 798 217504 814 99118 1345 2023-07-14T19:29:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:52 smithi133 ceph-mon[128894]: pgmap v43: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 217 op/s 2023-07-14T19:29:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:52 smithi082 ceph-mon[127461]: pgmap v43: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 217 op/s 2023-07-14T19:29:53.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:52 smithi161 ceph-mon[129184]: pgmap v43: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 217 op/s 2023-07-14T19:29:55.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:54 smithi133 ceph-mon[128894]: pgmap v44: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 210 op/s 2023-07-14T19:29:55.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:54 smithi082 ceph-mon[127461]: pgmap v44: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 210 op/s 2023-07-14T19:29:55.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:54 smithi161 ceph-mon[129184]: pgmap v44: 97 pgs: 97 active+clean; 52 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 210 op/s 2023-07-14T19:29:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:56 smithi133 ceph-mon[128894]: pgmap v45: 97 pgs: 97 active+clean; 59 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 280 op/s 2023-07-14T19:29:57.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:56 smithi082 ceph-mon[127461]: pgmap v45: 97 pgs: 97 active+clean; 59 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 280 op/s 2023-07-14T19:29:57.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:56 smithi161 ceph-mon[129184]: pgmap v45: 97 pgs: 97 active+clean; 59 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 280 op/s 2023-07-14T19:29:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 -- 172.21.15.82:0/4002391031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 msgr2=0x7f51d010b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 --2- 172.21.15.82:0/4002391031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d010b9d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f51c800b0d0 tx=0x7f51c800b3e0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 -- 172.21.15.82:0/4002391031 shutdown_connections 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 --2- 172.21.15.82:0/4002391031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d010b9d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 --2- 172.21.15.82:0/4002391031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 0x7f51d00ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 --2- 172.21.15.82:0/4002391031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d01003b0 0x7f51d0104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 -- 172.21.15.82:0/4002391031 >> 172.21.15.82:0/4002391031 conn(0x7f51d0074cc0 msgr2=0x7f51d00750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 -- 172.21.15.82:0/4002391031 shutdown_connections 2023-07-14T19:29:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.557+0000 7f51d53bd700 1 -- 172.21.15.82:0/4002391031 wait complete. 2023-07-14T19:29:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 Processor -- start 2023-07-14T19:29:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 -- start start 2023-07-14T19:29:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 0x7f51d019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d01003b0 0x7f51d019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d0194b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01951a0 con 0x7f51d0104fe0 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0195310 con 0x7f51d00ffa10 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51d53bd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d01954d0 con 0x7f51d01003b0 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d0194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51ceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 0x7f51d019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d0194b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43570/0 (socket says 172.21.15.82:43570) 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 -- 172.21.15.82:0/419159587 learned_addr learned my addr 172.21.15.82:0/419159587 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d01003b0 0x7f51d019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 -- 172.21.15.82:0/419159587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d01003b0 msgr2=0x7f51d019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d01003b0 0x7f51d019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 -- 172.21.15.82:0/419159587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 msgr2=0x7f51d019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 0x7f51d019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51cf7fe700 1 -- 172.21.15.82:0/419159587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8009d00 con 0x7f51d0104fe0 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.558+0000 7f51ceffd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 0x7f51d019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.559+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d0194b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51c8007870 tx=0x7f51c80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.559+0000 7f51b7fff700 1 -- 172.21.15.82:0/419159587 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800e040 con 0x7f51d0104fe0 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.559+0000 7f51b7fff700 1 -- 172.21.15.82:0/419159587 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c8005070 con 0x7f51d0104fe0 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.559+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d0195750 con 0x7f51d0104fe0 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.559+0000 7f51b7fff700 1 -- 172.21.15.82:0/419159587 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c801b6a0 con 0x7f51d0104fe0 2023-07-14T19:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.559+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d01a1cc0 con 0x7f51d0104fe0 2023-07-14T19:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.561+0000 7f51b7fff700 1 -- 172.21.15.82:0/419159587 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f51c80051e0 con 0x7f51d0104fe0 2023-07-14T19:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.562+0000 7f51b7fff700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f51b80a13e0 0x7f51b80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.562+0000 7f51ceffd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f51b80a13e0 0x7f51b80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.562+0000 7f51b7fff700 1 -- 172.21.15.82:0/419159587 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f51c80ce2a0 con 0x7f51d0104fe0 2023-07-14T19:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f51b80a13e0 0x7f51b80a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009cb0 tx=0x7f51c0009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 Processor -- start 2023-07-14T19:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 -- start start 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e85266b0 0x5653e86cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 0x5653e86cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e85cfc50 con 0x5653e86cbcc0 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e86229e0 con 0x5653e85266b0 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51e1555b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e861be70 con 0x5653e86d6a40 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ce7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 0x5653e86cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e85266b0 0x5653e86cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ce7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 0x5653e86cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43576/0 (socket says 172.21.15.82:43576) 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ce7fc700 1 -- 172.21.15.82:0/1250336855 learned_addr learned my addr 172.21.15.82:0/1250336855 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 -- 172.21.15.82:0/1250336855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 msgr2=0x5653e86d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 -- 172.21.15.82:0/1250336855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 msgr2=0x5653e86cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 0x5653e86cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ceffd700 1 -- 172.21.15.82:0/1250336855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653e8449d80 con 0x5653e85266b0 2023-07-14T19:29:58.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.563+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 0x5653e86cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51ceffd700 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e85266b0 0x5653e86cb780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51c0011f80 tx=0x7f51c0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3566b2a0fd543e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51b57fa700 1 -- 172.21.15.82:0/1250336855 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0009ea0 con 0x5653e85266b0 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51b57fa700 1 -- 172.21.15.82:0/1250336855 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c0003b40 con 0x5653e85266b0 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51b57fa700 1 -- 172.21.15.82:0/1250336855 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0012be0 con 0x5653e85266b0 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 -- 172.21.15.82:0/1250336855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e85266b0 msgr2=0x5653e86cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e85266b0 0x5653e86cb780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51c0011f80 tx=0x7f51c0014000 comp rx=0 tx=0).stop 2023-07-14T19:29:58.571 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 -- 172.21.15.82:0/1250336855 shutdown_connections 2023-07-14T19:29:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e85266b0 0x5653e86cb780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 --2- 172.21.15.82:0/1250336855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86cbcc0 0x5653e86cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.564+0000 7f51e1555b80 1 -- 172.21.15.82:0/1250336855 >> 172.21.15.82:0/1250336855 conn(0x5653e86c03d0 msgr2=0x5653e86c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:58.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 -- 172.21.15.82:0/1250336855 shutdown_connections 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 -- 172.21.15.82:0/1250336855 wait complete. 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 -- start start 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 Processor -- start 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86cbcc0 0x5653e86d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86d6a40 0x5653e86d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e85bba60 con 0x5653e85266b0 2023-07-14T19:29:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e85bb760 con 0x5653e86d6a40 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e849f8c0 con 0x5653e86cbcc0 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51e1555b80 10 client.0 _ll_register_callbacks cb 0x5653e85b9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51ceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51ceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86d64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43580/0 (socket says 172.21.15.82:43580) 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51ceffd700 1 -- 172.21.15.82:0/3475801766 learned_addr learned my addr 172.21.15.82:0/3475801766 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.565+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86d6a40 0x5653e86d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86cbcc0 0x5653e86d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ceffd700 1 -- 172.21.15.82:0/3475801766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86cbcc0 msgr2=0x5653e86d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ceffd700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86cbcc0 0x5653e86d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ceffd700 1 -- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86d6a40 msgr2=0x5653e86d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ceffd700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86d6a40 0x5653e86d3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ceffd700 1 -- 172.21.15.82:0/3475801766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c400b860 con 0x5653e85266b0 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86d6a40 0x5653e86d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51ceffd700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86d64b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f51c0013f20 tx=0x7f51c0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3532003f76954ab8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51b2ff5700 1 -- 172.21.15.82:0/3475801766 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0012bd0 con 0x5653e85266b0 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51b2ff5700 1 -- 172.21.15.82:0/3475801766 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f51c0018070 con 0x5653e85266b0 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653e8575490 con 0x5653e85266b0 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51b2ff5700 1 -- 172.21.15.82:0/3475801766 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c0003950 con 0x5653e85266b0 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51e1555b80 10 client.15801 fetch_fsmap learned FSMap version 29 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f51c0011c60 con 0x5653e85266b0 2023-07-14T19:29:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.566+0000 7f51b2ff5700 1 -- 172.21.15.82:0/3475801766 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653e8575490 con 0x5653e85266b0 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51b2ff5700 1 -- 172.21.15.82:0/3475801766 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f51c0004820 con 0x5653e85266b0 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51e1555b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51e1555b80 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51e1555b80 20 client.15801 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51e1555b80 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e86e65d0 0x5653e86e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51e1555b80 4 client.15801 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653e8699d40 con 0x5653e86e65d0 2023-07-14T19:29:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e86e65d0 0x5653e86e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51b2ff5700 10 client.15801 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.567+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e86e65d0 0x5653e86e89e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25292f5ad14cf28 server_cookie=b200fbc77522a1f2 in_seq=0 out_seq=0 2023-07-14T19:29:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51b2ff5700 1 -- 172.21.15.82:0/3475801766 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5653e8699d40 con 0x5653e86e65d0 2023-07-14T19:29:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51b2ff5700 10 client.15801 handle_command_reply: tid=0 2023-07-14T19:29:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 client.15801 shutdown 2023-07-14T19:29:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 msgr2=0x5653e86d64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86d64b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f51c0013f20 tx=0x7f51c0003680 comp rx=0 tx=0).stop 2023-07-14T19:29:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 shutdown_connections 2023-07-14T19:29:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86d6a40 0x5653e86d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86d64b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e86e65d0 0x5653e86e89e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 --2- 172.21.15.82:0/3475801766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86cbcc0 0x5653e86d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51b2ff5700 0 client.15801 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.568+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 >> 172.21.15.82:0/3475801766 conn(0x5653e86c03d0 msgr2=0x5653e86cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.569+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 shutdown_connections 2023-07-14T19:29:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.569+0000 7f51e1555b80 1 -- 172.21.15.82:0/3475801766 wait complete. 2023-07-14T19:29:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.569+0000 7f51e1555b80 20 client.15801 trim_cache size 0 max 16384 2023-07-14T19:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 Processor -- start 2023-07-14T19:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 -- start start 2023-07-14T19:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e86cc710 con 0x5653e85266b0 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e86d93c0 con 0x5653e86cbcc0 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51e1555b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e86cf3f0 con 0x5653e86d6a40 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86dd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43588/0 (socket says 172.21.15.82:43588) 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 -- 172.21.15.82:0/2613985005 learned_addr learned my addr 172.21.15.82:0/2613985005 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 -- 172.21.15.82:0/2613985005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 msgr2=0x5653e86e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 -- 172.21.15.82:0/2613985005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 msgr2=0x5653e86e1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86e1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 -- 172.21.15.82:0/2613985005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653e8432c00 con 0x5653e85266b0 2023-07-14T19:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.584+0000 7f51ceffd700 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86dd610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51c0012790 tx=0x7f51c0004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6fe8b4d06ea6aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51b5ffb700 1 -- 172.21.15.82:0/2613985005 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00039a0 con 0x5653e85266b0 2023-07-14T19:29:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51b5ffb700 1 -- 172.21.15.82:0/2613985005 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c0038070 con 0x5653e85266b0 2023-07-14T19:29:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51b5ffb700 1 -- 172.21.15.82:0/2613985005 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00050d0 con 0x5653e85266b0 2023-07-14T19:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 -- 172.21.15.82:0/2613985005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 msgr2=0x5653e86dd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86dd610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51c0012790 tx=0x7f51c0004d20 comp rx=0 tx=0).stop 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 -- 172.21.15.82:0/2613985005 shutdown_connections 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e86d6a40 0x5653e86e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e85266b0 0x5653e86dd610 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 --2- 172.21.15.82:0/2613985005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86e1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 -- 172.21.15.82:0/2613985005 >> 172.21.15.82:0/2613985005 conn(0x5653e86c03d0 msgr2=0x5653e86cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 -- 172.21.15.82:0/2613985005 shutdown_connections 2023-07-14T19:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.585+0000 7f51e1555b80 1 -- 172.21.15.82:0/2613985005 wait complete. 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 -- start start 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 Processor -- start 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e85266b0 0x5653e86db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 0x5653e86dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e86dd940 con 0x5653e86d6a40 2023-07-14T19:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e86ddab0 con 0x5653e86cbcc0 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653e87392b0 con 0x5653e85266b0 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51e1555b80 10 client.0 _ll_register_callbacks cb 0x7f51c0023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 0x5653e86dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 0x5653e86dc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43598/0 (socket says 172.21.15.82:43598) 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 -- 172.21.15.82:0/4066451965 learned_addr learned my addr 172.21.15.82:0/4066451965 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51ce7fc700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51ceffd700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e85266b0 0x5653e86db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 -- 172.21.15.82:0/4066451965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e85266b0 msgr2=0x5653e86db5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e85266b0 0x5653e86db5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 -- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 msgr2=0x5653e86dbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86dbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 -- 172.21.15.82:0/4066451965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c0011c60 con 0x5653e86d6a40 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51ceffd700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e85266b0 0x5653e86db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51cf7fe700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 0x5653e86dc070 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51c8097c50 tx=0x7f51c809b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce3e9da9a8e2099 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.586+0000 7f51b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:29:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51b2ff5700 1 -- 172.21.15.82:0/4066451965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c80a10b0 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51b2ff5700 1 -- 172.21.15.82:0/4066451965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f51c80054e0 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51c0003b40 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51b2ff5700 1 -- 172.21.15.82:0/4066451965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c809b1e0 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 10 client.15807 fetch_fsmap learned FSMap version 29 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5653e84701f0 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51b2ff5700 1 -- 172.21.15.82:0/4066451965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c0003b40 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51b2ff5700 1 -- 172.21.15.82:0/4066451965 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f51c80a3230 con 0x5653e86d6a40 2023-07-14T19:29:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 10 client.15807 fetch_fsmap finished waiting for FSMap version 29 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 20 client.15807 populate_metadata read hostname 'smithi082' 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e873a9a0 0x5653e873cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 4 client.15807 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5653e8614e50 con 0x5653e873a9a0 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.587+0000 7f51ceffd700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e873a9a0 0x5653e873cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:29:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.588+0000 7f51b2ff5700 10 client.15807 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.588+0000 7f51ceffd700 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e873a9a0 0x5653e873cdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667e111aa3333289 server_cookie=60fad41d630dd31a in_seq=0 out_seq=0 2023-07-14T19:29:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51b2ff5700 1 -- 172.21.15.82:0/4066451965 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5653e8614e50 con 0x5653e873a9a0 2023-07-14T19:29:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51b2ff5700 10 client.15807 handle_command_reply: tid=0 2023-07-14T19:29:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 client.15807 shutdown 2023-07-14T19:29:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 msgr2=0x5653e86dc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 0x5653e86dc070 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51c8097c50 tx=0x7f51c809b0a0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 shutdown_connections 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653e86d6a40 0x5653e86dc070 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5653e873a9a0 0x5653e873cdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653e85266b0 0x5653e86db5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 --2- 172.21.15.82:0/4066451965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653e86cbcc0 0x5653e86dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51b2ff5700 0 client.15807 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:29:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 >> 172.21.15.82:0/4066451965 conn(0x5653e86c03d0 msgr2=0x5653e86e1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 shutdown_connections 2023-07-14T19:29:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.686+0000 7f51e1555b80 1 -- 172.21.15.82:0/4066451965 wait complete. 2023-07-14T19:29:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51e1555b80 20 client.15807 trim_cache size 0 max 16384 2023-07-14T19:29:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:29:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:29:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:29:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:29:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f51b80a13e0 msgr2=0x7f51b80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f51b80a13e0 0x7f51b80a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009cb0 tx=0x7f51c0009260 comp rx=0 tx=0).stop 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 msgr2=0x7f51d0194b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d0194b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51c8007870 tx=0x7f51c80078a0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 shutdown_connections 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f51b80a13e0 0x7f51b80a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51d0104fe0 0x7f51d0194b30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51d00ffa10 0x7f51d019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 --2- 172.21.15.82:0/419159587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d01003b0 0x7f51d019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:29:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 >> 172.21.15.82:0/419159587 conn(0x7f51d0074cc0 msgr2=0x7f51d01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:29:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 shutdown_connections 2023-07-14T19:29:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:29:58.687+0000 7f51d53bd700 1 -- 172.21.15.82:0/419159587 wait complete. 2023-07-14T19:29:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:58 smithi133 ceph-mon[128894]: pgmap v46: 97 pgs: 97 active+clean; 59 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 245 op/s 2023-07-14T19:29:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:29:58 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:58 smithi082 ceph-mon[127461]: pgmap v46: 97 pgs: 97 active+clean; 59 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 245 op/s 2023-07-14T19:29:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:29:58 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:29:59.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:58 smithi161 ceph-mon[129184]: pgmap v46: 97 pgs: 97 active+clean; 59 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 245 op/s 2023-07-14T19:29:59.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:29:58 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:00.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:29:59.998+0000 7f9a2dc83700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-14T19:30:00.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:29:59.998+0000 7f9a2dc83700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-14T19:30:00.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:29:59.998+0000 7f9a2dc83700 -1 log_channel(cluster) log [ERR] : mds.h(mds.1): Metadata damage detected 2023-07-14T19:30:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:00 smithi133 ceph-mon[128894]: pgmap v47: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2023-07-14T19:30:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:00 smithi133 ceph-mon[128894]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-14T19:30:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:00 smithi133 ceph-mon[128894]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-14T19:30:01.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:00 smithi133 ceph-mon[128894]: mds.h(mds.1): Metadata damage detected 2023-07-14T19:30:01.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-mon[127461]: pgmap v47: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2023-07-14T19:30:01.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-mon[127461]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-14T19:30:01.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-mon[127461]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-14T19:30:01.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:00 smithi082 ceph-mon[127461]: mds.h(mds.1): Metadata damage detected 2023-07-14T19:30:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:00 smithi161 ceph-mon[129184]: pgmap v47: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2023-07-14T19:30:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:00 smithi161 ceph-mon[129184]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-07-14T19:30:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:00 smithi161 ceph-mon[129184]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-07-14T19:30:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:00 smithi161 ceph-mon[129184]: mds.h(mds.1): Metadata damage detected 2023-07-14T19:30:02.170 INFO:tasks.workunit.client.0.smithi082.stdout: 26 422025 473 360367 718 190471 1420 2023-07-14T19:30:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:02 smithi133 ceph-mon[128894]: pgmap v48: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-14T19:30:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:02 smithi082 ceph-mon[127461]: pgmap v48: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-14T19:30:03.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:02 smithi161 ceph-mon[129184]: pgmap v48: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-14T19:30:04.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:03 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:04.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:03 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:04.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:03 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:05.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:04 smithi082 ceph-mon[127461]: pgmap v49: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-14T19:30:05.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:04 smithi161 ceph-mon[129184]: pgmap v49: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-14T19:30:05.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:04 smithi133 ceph-mon[128894]: pgmap v49: 97 pgs: 97 active+clean; 67 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 105 op/s 2023-07-14T19:30:06.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:05 smithi082 ceph-mon[127461]: mgrmap e24: x(active, since 91s), standbys: y, z 2023-07-14T19:30:06.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:05 smithi161 ceph-mon[129184]: mgrmap e24: x(active, since 91s), standbys: y, z 2023-07-14T19:30:06.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:05 smithi133 ceph-mon[128894]: mgrmap e24: x(active, since 91s), standbys: y, z 2023-07-14T19:30:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:06 smithi082 ceph-mon[127461]: pgmap v50: 97 pgs: 97 active+clean; 73 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2023-07-14T19:30:07.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:06 smithi161 ceph-mon[129184]: pgmap v50: 97 pgs: 97 active+clean; 73 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2023-07-14T19:30:07.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:06 smithi133 ceph-mon[128894]: pgmap v50: 97 pgs: 97 active+clean; 73 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2023-07-14T19:30:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:08 smithi082 ceph-mon[127461]: pgmap v51: 97 pgs: 97 active+clean; 73 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2023-07-14T19:30:09.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:08 smithi161 ceph-mon[129184]: pgmap v51: 97 pgs: 97 active+clean; 73 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2023-07-14T19:30:09.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:08 smithi133 ceph-mon[128894]: pgmap v51: 97 pgs: 97 active+clean; 73 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 87 op/s 2023-07-14T19:30:11.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:10 smithi082 ceph-mon[127461]: pgmap v52: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 110 op/s 2023-07-14T19:30:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:10 smithi161 ceph-mon[129184]: pgmap v52: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 110 op/s 2023-07-14T19:30:11.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:10 smithi133 ceph-mon[128894]: pgmap v52: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 110 op/s 2023-07-14T19:30:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 -- 172.21.15.82:0/1992744436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d241008b0 msgr2=0x7f3d24104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 --2- 172.21.15.82:0/1992744436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d241008b0 0x7f3d24104870 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d200099c0 tx=0x7f3d20009cd0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 -- 172.21.15.82:0/1992744436 shutdown_connections 2023-07-14T19:30:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 --2- 172.21.15.82:0/1992744436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d24104f50 0x7f3d2410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 --2- 172.21.15.82:0/1992744436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d240fff10 0x7f3d241002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 --2- 172.21.15.82:0/1992744436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d241008b0 0x7f3d24104870 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 -- 172.21.15.82:0/1992744436 >> 172.21.15.82:0/1992744436 conn(0x7f3d240fb820 msgr2=0x7f3d240fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 -- 172.21.15.82:0/1992744436 shutdown_connections 2023-07-14T19:30:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 -- 172.21.15.82:0/1992744436 wait complete. 2023-07-14T19:30:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 Processor -- start 2023-07-14T19:30:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.799+0000 7f3d2b370700 1 -- start start 2023-07-14T19:30:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 0x7f3d24195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d241008b0 0x7f3d241962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d24104f50 0x7f3d2419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419af90 con 0x7f3d240fff10 2023-07-14T19:30:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419b100 con 0x7f3d24104f50 2023-07-14T19:30:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2419b2c0 con 0x7f3d241008b0 2023-07-14T19:30:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 0x7f3d24195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 0x7f3d24195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43608/0 (socket says 172.21.15.82:43608) 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 -- 172.21.15.82:0/2897650818 learned_addr learned my addr 172.21.15.82:0/2897650818 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2990d700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d24104f50 0x7f3d2419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d241008b0 0x7f3d241962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 -- 172.21.15.82:0/2897650818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d241008b0 msgr2=0x7f3d241962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d241008b0 0x7f3d241962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 -- 172.21.15.82:0/2897650818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d24104f50 msgr2=0x7f3d2419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d24104f50 0x7f3d2419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 -- 172.21.15.82:0/2897650818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200095f0 con 0x7f3d240fff10 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 0x7f3d24195da0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9a0 tx=0x7f3d1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d1a7fc700 1 -- 172.21.15.82:0/2897650818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14009790 con 0x7f3d240fff10 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d1a7fc700 1 -- 172.21.15.82:0/2897650818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d14010430 con 0x7f3d240fff10 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d1a7fc700 1 -- 172.21.15.82:0/2897650818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14018680 con 0x7f3d240fff10 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.800+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2419b5a0 con 0x7f3d240fff10 2023-07-14T19:30:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.801+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a2de0 con 0x7f3d240fff10 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.803+0000 7f3d1a7fc700 1 -- 172.21.15.82:0/2897650818 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3d140105a0 con 0x7f3d240fff10 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.803+0000 7f3d1a7fc700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3d100a1340 0x7f3d100a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.804+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3d100a1340 0x7f3d100a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.804+0000 7f3d1a7fc700 1 -- 172.21.15.82:0/2897650818 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f3d14014070 con 0x7f3d240fff10 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.804+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3d100a1340 0x7f3d100a37f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d200098e0 tx=0x7f3d20000bc0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.804+0000 7f3d37508b80 1 Processor -- start 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 -- start start 2023-07-14T19:30:11.817 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbda90 0x55f364bb79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bb7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364abcc50 con 0x55f364bbdf00 2023-07-14T19:30:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364b0f9e0 con 0x55f364a136b0 2023-07-14T19:30:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d37508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364b08e70 con 0x55f364bbda90 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2990d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bb7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bbd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:39964/0 (socket says 172.21.15.82:39964) 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2990d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bb7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43620/0 (socket says 172.21.15.82:43620) 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 -- 172.21.15.82:0/2100414281 learned_addr learned my addr 172.21.15.82:0/2100414281 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbda90 0x55f364bb79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 -- 172.21.15.82:0/2100414281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbda90 msgr2=0x55f364bb79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbda90 0x55f364bb79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 -- 172.21.15.82:0/2100414281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 msgr2=0x55f364bb7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bb7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 -- 172.21.15.82:0/2100414281 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f364936d80 con 0x55f364a136b0 2023-07-14T19:30:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2990d700 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bb7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.805+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bbd550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d140a1e80 tx=0x7f3d140999b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=641b247153800a97 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d0f7fe700 1 -- 172.21.15.82:0/2100414281 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140a23b0 con 0x55f364a136b0 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d0f7fe700 1 -- 172.21.15.82:0/2100414281 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d14010880 con 0x55f364a136b0 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d0f7fe700 1 -- 172.21.15.82:0/2100414281 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d140a1f20 con 0x55f364a136b0 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 -- 172.21.15.82:0/2100414281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 msgr2=0x55f364bbd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bbd550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d140a1e80 tx=0x7f3d140999b0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 -- 172.21.15.82:0/2100414281 shutdown_connections 2023-07-14T19:30:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bb7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bbd550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2100414281 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbda90 0x55f364bb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 -- 172.21.15.82:0/2100414281 >> 172.21.15.82:0/2100414281 conn(0x55f364bad3d0 msgr2=0x55f364bb2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 -- 172.21.15.82:0/2100414281 shutdown_connections 2023-07-14T19:30:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.806+0000 7f3d37508b80 1 -- 172.21.15.82:0/2100414281 wait complete. 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 -- start start 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 Processor -- start 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bc6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbda90 0x55f364bc73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bc10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364aa8a60 con 0x55f364bbdf00 2023-07-14T19:30:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364aa8760 con 0x55f364bbda90 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36498c8c0 con 0x55f364a136b0 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d37508b80 10 client.0 _ll_register_callbacks cb 0x55f364aa6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d2990d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bc10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d2890b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbda90 0x55f364bc73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d2990d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bc10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43626/0 (socket says 172.21.15.82:43626) 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d2990d700 1 -- 172.21.15.82:0/76024304 learned_addr learned my addr 172.21.15.82:0/76024304 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.807+0000 7f3d2910c700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bc6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2990d700 1 -- 172.21.15.82:0/76024304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 msgr2=0x55f364bc6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2990d700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bc6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2990d700 1 -- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbda90 msgr2=0x55f364bc73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2990d700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbda90 0x55f364bc73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2990d700 1 -- 172.21.15.82:0/76024304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c009ba0 con 0x55f364bbdf00 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2910c700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bc6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d2990d700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bc10c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00ae60 tx=0x7f3d1c011490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd0463988267793 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d0cff9700 1 -- 172.21.15.82:0/76024304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c010040 con 0x55f364bbdf00 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d0cff9700 1 -- 172.21.15.82:0/76024304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f3d1c006e40 con 0x55f364bbdf00 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f364a62490 con 0x55f364bbdf00 2023-07-14T19:30:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d0cff9700 1 -- 172.21.15.82:0/76024304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d1c011ca0 con 0x55f364bbdf00 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d37508b80 10 client.15819 fetch_fsmap learned FSMap version 30 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f3d241a2de0 con 0x55f364bbdf00 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d0cff9700 1 -- 172.21.15.82:0/76024304 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f364a62490 con 0x55f364bbdf00 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d0cff9700 1 -- 172.21.15.82:0/76024304 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3d1c008030 con 0x55f364bbdf00 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d37508b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.808+0000 7f3d37508b80 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d37508b80 20 client.15819 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d37508b80 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364bd13e0 0x55f364bd37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d37508b80 4 client.15819 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:30:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f364b86d40 con 0x55f364bd13e0 2023-07-14T19:30:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d2910c700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364bd13e0 0x55f364bd37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d2910c700 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364bd13e0 0x55f364bd37f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b401369aa98dd7 server_cookie=9b79eb86d4c6950c in_seq=0 out_seq=0 2023-07-14T19:30:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.809+0000 7f3d0cff9700 10 client.15819 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d0cff9700 1 -- 172.21.15.82:0/76024304 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f364b86d40 con 0x55f364bd13e0 2023-07-14T19:30:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d0cff9700 10 client.15819 handle_command_reply: tid=0 2023-07-14T19:30:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 client.15819 shutdown 2023-07-14T19:30:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 msgr2=0x55f364bc10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bc10c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00ae60 tx=0x7f3d1c011490 comp rx=0 tx=0).stop 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 shutdown_connections 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbdf00 0x55f364bc10c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364bd13e0 0x55f364bd37f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bc6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d0cff9700 0 client.15819 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 --2- 172.21.15.82:0/76024304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbda90 0x55f364bc73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 >> 172.21.15.82:0/76024304 conn(0x55f364bad3d0 msgr2=0x55f364bb1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 shutdown_connections 2023-07-14T19:30:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 1 -- 172.21.15.82:0/76024304 wait complete. 2023-07-14T19:30:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.810+0000 7f3d37508b80 20 client.15819 trim_cache size 0 max 16384 2023-07-14T19:30:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 Processor -- start 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 -- start start 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bb23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bbc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbdf00 0x55f364bc0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364bce9b0 con 0x55f364bbda90 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364bd85b0 con 0x55f364bbdf00 2023-07-14T19:30:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.827+0000 7f3d37508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364bb8970 con 0x55f364a136b0 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bbc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bbc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43634/0 (socket says 172.21.15.82:43634) 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 -- 172.21.15.82:0/2067957673 learned_addr learned my addr 172.21.15.82:0/2067957673 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2990d700 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbdf00 0x55f364bc0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2910c700 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bb23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 -- 172.21.15.82:0/2067957673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 msgr2=0x55f364bb23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bb23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 -- 172.21.15.82:0/2067957673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbdf00 msgr2=0x55f364bc0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbdf00 0x55f364bc0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 -- 172.21.15.82:0/2067957673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f36491fc00 con 0x55f364bbda90 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d2890b700 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bbc860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3d200099c0 tx=0x7f3d20022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00f8864d74ce493 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d0ffff700 1 -- 172.21.15.82:0/2067957673 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2002c030 con 0x55f364bbda90 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d0ffff700 1 -- 172.21.15.82:0/2067957673 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d2002d040 con 0x55f364bbda90 2023-07-14T19:30:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.828+0000 7f3d0ffff700 1 -- 172.21.15.82:0/2067957673 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000f6c0 con 0x55f364bbda90 2023-07-14T19:30:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 -- 172.21.15.82:0/2067957673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 msgr2=0x55f364bbc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bbc860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3d200099c0 tx=0x7f3d20022000 comp rx=0 tx=0).stop 2023-07-14T19:30:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 -- 172.21.15.82:0/2067957673 shutdown_connections 2023-07-14T19:30:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364bbdf00 0x55f364bc0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364a136b0 0x55f364bb23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 --2- 172.21.15.82:0/2067957673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bbc860 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 -- 172.21.15.82:0/2067957673 >> 172.21.15.82:0/2067957673 conn(0x55f364bad3d0 msgr2=0x55f364bbc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 -- 172.21.15.82:0/2067957673 shutdown_connections 2023-07-14T19:30:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 -- 172.21.15.82:0/2067957673 wait complete. 2023-07-14T19:30:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 -- start start 2023-07-14T19:30:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.829+0000 7f3d37508b80 1 Processor -- start 2023-07-14T19:30:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bc7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bc54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbdf00 0x55f364bc59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364bc60d0 con 0x55f364bbda90 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364bc6240 con 0x55f364a136b0 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f364c29460 con 0x55f364bbdf00 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d37508b80 10 client.0 _ll_register_callbacks cb 0x7f3d1c00a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d2890b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bc54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d2910c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d2890b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bc54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43638/0 (socket says 172.21.15.82:43638) 2023-07-14T19:30:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d2890b700 1 -- 172.21.15.82:0/1129662729 learned_addr learned my addr 172.21.15.82:0/1129662729 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.830+0000 7f3d2990d700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbdf00 0x55f364bc59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2890b700 1 -- 172.21.15.82:0/1129662729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbdf00 msgr2=0x55f364bc59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2890b700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbdf00 0x55f364bc59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2890b700 1 -- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 msgr2=0x55f364bc7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2890b700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bc7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2890b700 1 -- 172.21.15.82:0/1129662729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2410d360 con 0x55f364bbda90 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2990d700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbdf00 0x55f364bc59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2910c700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bc7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d2890b700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bc54b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3d20003aa0 tx=0x7f3d20003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35c8510f0d99335 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d0cff9700 1 -- 172.21.15.82:0/1129662729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2002c030 con 0x55f364bbda90 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d0cff9700 1 -- 172.21.15.82:0/1129662729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f3d20004400 con 0x55f364bbda90 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d14010880 con 0x55f364bbda90 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.831+0000 7f3d0cff9700 1 -- 172.21.15.82:0/1129662729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d200045c0 con 0x55f364bbda90 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 10 client.15825 fetch_fsmap learned FSMap version 30 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f36495d1f0 con 0x55f364bbda90 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d0cff9700 1 -- 172.21.15.82:0/1129662729 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d20049020 con 0x55f364bbda90 2023-07-14T19:30:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d0cff9700 1 -- 172.21.15.82:0/1129662729 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3d2003b5f0 con 0x55f364bbda90 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 20 client.15825 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364c2a620 0x55f364c2ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 4 client.15825 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f364b01e50 con 0x55f364c2a620 2023-07-14T19:30:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.832+0000 7f3d2910c700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364c2a620 0x55f364c2ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.833+0000 7f3d0cff9700 10 client.15825 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:11.833+0000 7f3d2910c700 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364c2a620 0x55f364c2ca30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1934882b2b18727 server_cookie=ba15c14bec369009 in_seq=0 out_seq=0 2023-07-14T19:30:12.170 INFO:tasks.workunit.client.0.smithi082.stdout: 58 15433 1715 13354 1441 9089 256 2023-07-14T19:30:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d0cff9700 1 -- 172.21.15.82:0/1129662729 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f364b01e50 con 0x55f364c2a620 2023-07-14T19:30:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d0cff9700 10 client.15825 handle_command_reply: tid=0 2023-07-14T19:30:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 client.15825 shutdown 2023-07-14T19:30:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 msgr2=0x55f364bc54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bc54b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3d20003aa0 tx=0x7f3d20003e40 comp rx=0 tx=0).stop 2023-07-14T19:30:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 shutdown_connections 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f364c2a620 0x55f364c2ca30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f364bbdf00 0x55f364bc59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d0cff9700 0 client.15825 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f364a136b0 0x55f364bc7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 --2- 172.21.15.82:0/1129662729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f364bbda90 0x55f364bc54b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 >> 172.21.15.82:0/1129662729 conn(0x55f364bad3d0 msgr2=0x55f364bbc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 shutdown_connections 2023-07-14T19:30:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 1 -- 172.21.15.82:0/1129662729 wait complete. 2023-07-14T19:30:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.034+0000 7f3d37508b80 20 client.15825 trim_cache size 0 max 16384 2023-07-14T19:30:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:30:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:30:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:30:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:30:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3d100a1340 msgr2=0x7f3d100a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3d100a1340 0x7f3d100a37f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d200098e0 tx=0x7f3d20000bc0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 msgr2=0x7f3d24195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 0x7f3d24195da0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9a0 tx=0x7f3d1400ecb0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 shutdown_connections 2023-07-14T19:30:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3d100a1340 0x7f3d100a37f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d24104f50 0x7f3d2419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d240fff10 0x7f3d24195da0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 --2- 172.21.15.82:0/2897650818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3d241008b0 0x7f3d241962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 >> 172.21.15.82:0/2897650818 conn(0x7f3d240fb820 msgr2=0x7f3d24103890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 shutdown_connections 2023-07-14T19:30:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:13.035+0000 7f3d2b370700 1 -- 172.21.15.82:0/2897650818 wait complete. 2023-07-14T19:30:13.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:12 smithi082 ceph-mon[127461]: pgmap v53: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 75 op/s 2023-07-14T19:30:13.244 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:12 smithi161 ceph-mon[129184]: pgmap v53: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 75 op/s 2023-07-14T19:30:13.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:12 smithi133 ceph-mon[128894]: pgmap v53: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 75 op/s 2023-07-14T19:30:15.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:14 smithi082 ceph-mon[127461]: pgmap v54: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 75 op/s 2023-07-14T19:30:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:14 smithi161 ceph-mon[129184]: pgmap v54: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 75 op/s 2023-07-14T19:30:15.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:14 smithi133 ceph-mon[128894]: pgmap v54: 97 pgs: 97 active+clean; 84 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 75 op/s 2023-07-14T19:30:16.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:15 smithi161 ceph-mon[129184]: pgmap v55: 97 pgs: 97 active+clean; 244 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 874 op/s 2023-07-14T19:30:16.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:15 smithi133 ceph-mon[128894]: pgmap v55: 97 pgs: 97 active+clean; 244 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 874 op/s 2023-07-14T19:30:16.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:15 smithi082 ceph-mon[127461]: pgmap v55: 97 pgs: 97 active+clean; 244 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 15 MiB/s wr, 874 op/s 2023-07-14T19:30:18.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:18 smithi133 ceph-mon[128894]: pgmap v56: 97 pgs: 97 active+clean; 244 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 15 MiB/s wr, 821 op/s 2023-07-14T19:30:18.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:18 smithi082 ceph-mon[127461]: pgmap v56: 97 pgs: 97 active+clean; 244 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 15 MiB/s wr, 821 op/s 2023-07-14T19:30:18.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:18 smithi161 ceph-mon[129184]: pgmap v56: 97 pgs: 97 active+clean; 244 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 15 MiB/s wr, 821 op/s 2023-07-14T19:30:20.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:20 smithi133 ceph-mon[128894]: pgmap v57: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 875 op/s 2023-07-14T19:30:20.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:20 smithi082 ceph-mon[127461]: pgmap v57: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 875 op/s 2023-07-14T19:30:20.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:20 smithi161 ceph-mon[129184]: pgmap v57: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 16 MiB/s wr, 875 op/s 2023-07-14T19:30:22.170 INFO:tasks.workunit.client.0.smithi082.stdout: 64 20737 423 15178 235 5709 146 2023-07-14T19:30:22.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:22 smithi133 ceph-mon[128894]: pgmap v58: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 852 op/s 2023-07-14T19:30:22.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:22 smithi082 ceph-mon[127461]: pgmap v58: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 852 op/s 2023-07-14T19:30:22.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:22 smithi161 ceph-mon[129184]: pgmap v58: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 852 op/s 2023-07-14T19:30:24.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:24 smithi133 ceph-mon[128894]: pgmap v59: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 852 op/s 2023-07-14T19:30:24.967 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:24 smithi082 ceph-mon[127461]: pgmap v59: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 852 op/s 2023-07-14T19:30:24.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:24 smithi161 ceph-mon[129184]: pgmap v59: 97 pgs: 97 active+clean; 259 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 15 MiB/s wr, 852 op/s 2023-07-14T19:30:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 -- 172.21.15.82:0/166491536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438102a70 msgr2=0x7f8438106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 --2- 172.21.15.82:0/166491536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438102a70 0x7f8438106a30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f842c0099c0 tx=0x7f842c009cd0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 -- 172.21.15.82:0/166491536 shutdown_connections 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 --2- 172.21.15.82:0/166491536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8438107110 0x7f843810ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 --2- 172.21.15.82:0/166491536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84381020d0 0x7f84381024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 --2- 172.21.15.82:0/166491536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438102a70 0x7f8438106a30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 -- 172.21.15.82:0/166491536 >> 172.21.15.82:0/166491536 conn(0x7f84380fd960 msgr2=0x7f84380ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 -- 172.21.15.82:0/166491536 shutdown_connections 2023-07-14T19:30:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.178+0000 7f843e77e700 1 -- 172.21.15.82:0/166491536 wait complete. 2023-07-14T19:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.179+0000 7f843e77e700 1 Processor -- start 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.179+0000 7f843e77e700 1 -- start start 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843e77e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84381020d0 0x7f8438197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843e77e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 0x7f84381984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843e77e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438107110 0x7f843819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843e77e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843819d150 con 0x7f8438107110 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843e77e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843819d2c0 con 0x7f8438102a70 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843e77e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843819d480 con 0x7f84381020d0 2023-07-14T19:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 0x7f84381984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843cd1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438107110 0x7f843819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 0x7f84381984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40000/0 (socket says 172.21.15.82:40000) 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 -- 172.21.15.82:0/3482561871 learned_addr learned my addr 172.21.15.82:0/3482561871 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f8437fff700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84381020d0 0x7f8438197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 -- 172.21.15.82:0/3482561871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84381020d0 msgr2=0x7f8438197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84381020d0 0x7f8438197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 -- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438107110 msgr2=0x7f843819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438107110 0x7f843819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 -- 172.21.15.82:0/3482561871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0095f0 con 0x7f8438102a70 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f843cd1b700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438107110 0x7f843819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.180+0000 7f84377fe700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 0x7f84381984a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098e0 tx=0x7f842c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.181+0000 7f84357fa700 1 -- 172.21.15.82:0/3482561871 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c01d070 con 0x7f8438102a70 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.181+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843819d700 con 0x7f8438102a70 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.181+0000 7f84357fa700 1 -- 172.21.15.82:0/3482561871 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842c022470 con 0x7f8438102a70 2023-07-14T19:30:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.181+0000 7f84357fa700 1 -- 172.21.15.82:0/3482561871 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c017650 con 0x7f8438102a70 2023-07-14T19:30:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.181+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84381a4fa0 con 0x7f8438102a70 2023-07-14T19:30:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.183+0000 7f84357fa700 1 -- 172.21.15.82:0/3482561871 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f842c00fad0 con 0x7f8438102a70 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.184+0000 7f84357fa700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f84200a12c0 0x7f84200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.184+0000 7f84357fa700 1 -- 172.21.15.82:0/3482561871 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f842c0cd900 con 0x7f8438102a70 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.184+0000 7f8437fff700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f84200a12c0 0x7f84200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.184+0000 7f8437fff700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f84200a12c0 0x7f84200a3770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8428009c60 tx=0x7f8428009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.184+0000 7f844a916b80 1 Processor -- start 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.184+0000 7f844a916b80 1 -- start start 2023-07-14T19:30:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f844a916b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f844a916b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f844a916b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f844a916b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed5fec50 con 0x55d4ed6fff00 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f844a916b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed6519e0 con 0x55d4ed5556b0 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f844a916b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed64ae70 con 0x55d4ed6ffa90 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43662/0 (socket says 172.21.15.82:43662) 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 -- 172.21.15.82:0/75706651 learned_addr learned my addr 172.21.15.82:0/75706651 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f8437fff700 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f84377fe700 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 -- 172.21.15.82:0/75706651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6ffa90 msgr2=0x55d4ed6f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 -- 172.21.15.82:0/75706651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 msgr2=0x55d4ed6ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 -- 172.21.15.82:0/75706651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ed478d80 con 0x55d4ed6fff00 2023-07-14T19:30:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f843cd1b700 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f9f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8430003eb0 tx=0x7f8430009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c228c4d2fc8435a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f841e7fc700 1 -- 172.21.15.82:0/75706651 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300092d0 con 0x55d4ed6fff00 2023-07-14T19:30:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f841e7fc700 1 -- 172.21.15.82:0/75706651 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843000f040 con 0x55d4ed6fff00 2023-07-14T19:30:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.185+0000 7f841e7fc700 1 -- 172.21.15.82:0/75706651 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000e8b0 con 0x55d4ed6fff00 2023-07-14T19:30:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 -- 172.21.15.82:0/75706651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 msgr2=0x55d4ed6f9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f9f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8430003eb0 tx=0x7f8430009920 comp rx=0 tx=0).stop 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 -- 172.21.15.82:0/75706651 shutdown_connections 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f9f80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 --2- 172.21.15.82:0/75706651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 -- 172.21.15.82:0/75706651 >> 172.21.15.82:0/75706651 conn(0x55d4ed6ef3d0 msgr2=0x55d4ed6f4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 -- 172.21.15.82:0/75706651 shutdown_connections 2023-07-14T19:30:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.186+0000 7f844a916b80 1 -- 172.21.15.82:0/75706651 wait complete. 2023-07-14T19:30:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 -- start start 2023-07-14T19:30:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 Processor -- start 2023-07-14T19:30:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed705e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed7030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed703790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed5eaa60 con 0x55d4ed6ffa90 2023-07-14T19:30:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed5ea760 con 0x55d4ed6fff00 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed4ce8c0 con 0x55d4ed5556b0 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f844a916b80 10 client.0 _ll_register_callbacks cb 0x55d4ed5e8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed7030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed7030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43666/0 (socket says 172.21.15.82:43666) 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 -- 172.21.15.82:0/2666020994 learned_addr learned my addr 172.21.15.82:0/2666020994 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f8437fff700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed705e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f843cd1b700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed703790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 -- 172.21.15.82:0/2666020994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 msgr2=0x55d4ed705e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed705e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 -- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 msgr2=0x55d4ed703790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed703790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.187+0000 7f84377fe700 1 -- 172.21.15.82:0/2666020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8430003b90 con 0x55d4ed6ffa90 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84377fe700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed7030f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b580 tx=0x7f842c09aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f03b94e5f1f46b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84177fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:30:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84177fe700 1 -- 172.21.15.82:0/2666020994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c09eba0 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84177fe700 1 -- 172.21.15.82:0/2666020994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f842c0b3220 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4ed5a4490 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84177fe700 1 -- 172.21.15.82:0/2666020994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842c0a2eb0 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 10 client.15837 fetch_fsmap learned FSMap version 30 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d4ed461c00 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84177fe700 1 -- 172.21.15.82:0/2666020994 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842c0b7020 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f84177fe700 1 -- 172.21.15.82:0/2666020994 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f842c0a2a20 con 0x55d4ed6ffa90 2023-07-14T19:30:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 20 client.15837 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed71b7f0 0x55d4ed71dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 4 client.15837 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.188+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4ed6c8d40 con 0x55d4ed71b7f0 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.189+0000 7f8437fff700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed71b7f0 0x55d4ed71dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.189+0000 7f84177fe700 10 client.15837 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.189+0000 7f8437fff700 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed71b7f0 0x55d4ed71dc00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfe69e529bb16117 server_cookie=571df42d829d3d9c in_seq=0 out_seq=0 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f84177fe700 1 -- 172.21.15.82:0/2666020994 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55d4ed6c8d40 con 0x55d4ed71b7f0 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f84177fe700 10 client.15837 handle_command_reply: tid=0 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 client.15837 shutdown 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 msgr2=0x55d4ed7030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed7030f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b580 tx=0x7f842c09aaf0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 shutdown_connections 2023-07-14T19:30:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed703790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed71b7f0 0x55d4ed71dc00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed705e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f84177fe700 0 client.15837 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 --2- 172.21.15.82:0/2666020994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed7030f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 >> 172.21.15.82:0/2666020994 conn(0x55d4ed6ef3d0 msgr2=0x55d4ed6f3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 shutdown_connections 2023-07-14T19:30:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 1 -- 172.21.15.82:0/2666020994 wait complete. 2023-07-14T19:30:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.191+0000 7f844a916b80 20 client.15837 trim_cache size 0 max 16384 2023-07-14T19:30:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 Processor -- start 2023-07-14T19:30:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 -- start start 2023-07-14T19:30:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed712c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed7040a0 con 0x55d4ed6ffa90 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed6f5d70 con 0x55d4ed6fff00 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f844a916b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed6f1450 con 0x55d4ed5556b0 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43674/0 (socket says 172.21.15.82:43674) 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 -- 172.21.15.82:0/2072326115 learned_addr learned my addr 172.21.15.82:0/2072326115 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f8437fff700 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed712c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f843cd1b700 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 -- 172.21.15.82:0/2072326115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 msgr2=0x55d4ed712c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed712c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 -- 172.21.15.82:0/2072326115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 msgr2=0x55d4ed6f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 -- 172.21.15.82:0/2072326115 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4ed49f1f0 con 0x55d4ed6ffa90 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f8437fff700 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed712c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.207+0000 7f84377fe700 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fe8b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f842c097560 tx=0x7f842c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4e6692eb28aa52 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f841effd700 1 -- 172.21.15.82:0/2072326115 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0b3240 con 0x55d4ed6ffa90 2023-07-14T19:30:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f841effd700 1 -- 172.21.15.82:0/2072326115 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842c0b7030 con 0x55d4ed6ffa90 2023-07-14T19:30:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f841effd700 1 -- 172.21.15.82:0/2072326115 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c09a240 con 0x55d4ed6ffa90 2023-07-14T19:30:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 -- 172.21.15.82:0/2072326115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 msgr2=0x55d4ed6fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fe8b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f842c097560 tx=0x7f842c00ff90 comp rx=0 tx=0).stop 2023-07-14T19:30:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 -- 172.21.15.82:0/2072326115 shutdown_connections 2023-07-14T19:30:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed6fff00 0x55d4ed6f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed5556b0 0x55d4ed712c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 --2- 172.21.15.82:0/2072326115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fe8b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 -- 172.21.15.82:0/2072326115 >> 172.21.15.82:0/2072326115 conn(0x55d4ed6ef3d0 msgr2=0x55d4ed6fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 -- 172.21.15.82:0/2072326115 shutdown_connections 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.208+0000 7f844a916b80 1 -- 172.21.15.82:0/2072326115 wait complete. 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 -- start start 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 Processor -- start 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6fc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6fff00 0x55d4ed6fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed6fb5e0 con 0x55d4ed6ffa90 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed6fb750 con 0x55d4ed5556b0 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4ed6fb910 con 0x55d4ed6fff00 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.209+0000 7f844a916b80 10 client.0 _ll_register_callbacks cb 0x7f842c09a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40026/0 (socket says 172.21.15.82:40026) 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 -- 172.21.15.82:0/2661337774 learned_addr learned my addr 172.21.15.82:0/2661337774 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f84377fe700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f843cd1b700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6fff00 0x55d4ed6fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 -- 172.21.15.82:0/2661337774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6fff00 msgr2=0x55d4ed6fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6fff00 0x55d4ed6fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 -- 172.21.15.82:0/2661337774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 msgr2=0x55d4ed6fc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 -- 172.21.15.82:0/2661337774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8428017070 con 0x55d4ed5556b0 2023-07-14T19:30:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f8437fff700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6fc280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84280094f0 tx=0x7f84280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1f40d5d53c000 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f84177fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f84177fe700 1 -- 172.21.15.82:0/2661337774 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428022040 con 0x55d4ed5556b0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f84177fe700 1 -- 172.21.15.82:0/2661337774 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f842802be40 con 0x55d4ed5556b0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.210+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f843000f040 con 0x55d4ed5556b0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f84177fe700 1 -- 172.21.15.82:0/2661337774 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f842802b250 con 0x55d4ed5556b0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 10 client.25523 fetch_fsmap learned FSMap version 30 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d4ed461c00 con 0x55d4ed5556b0 2023-07-14T19:30:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f84177fe700 1 -- 172.21.15.82:0/2661337774 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843000f040 con 0x55d4ed5556b0 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f84177fe700 1 -- 172.21.15.82:0/2661337774 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8428034520 con 0x55d4ed5556b0 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 10 client.25523 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 20 client.25523 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed770470 0x55d4ed772880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 4 client.25523 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4ed643e50 con 0x55d4ed770470 2023-07-14T19:30:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.211+0000 7f84377fe700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed770470 0x55d4ed772880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.212+0000 7f84177fe700 10 client.25523 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.212+0000 7f84377fe700 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed770470 0x55d4ed772880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fc6b7f85fab027 server_cookie=d83bdfd7a4ddaabb in_seq=0 out_seq=0 2023-07-14T19:30:26.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:26 smithi133 ceph-mon[128894]: pgmap v60: 97 pgs: 97 active+clean; 268 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 957 op/s 2023-07-14T19:30:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f84177fe700 1 -- 172.21.15.82:0/2661337774 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4ed643e50 con 0x55d4ed770470 2023-07-14T19:30:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f84177fe700 10 client.25523 handle_command_reply: tid=0 2023-07-14T19:30:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 client.25523 shutdown 2023-07-14T19:30:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 msgr2=0x55d4ed6fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6fc280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84280094f0 tx=0x7f84280047c0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 shutdown_connections 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55d4ed770470 0x55d4ed772880 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d4ed6fff00 0x55d4ed6fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f84177fe700 0 client.25523 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4ed5556b0 0x55d4ed6fc280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 --2- 172.21.15.82:0/2661337774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4ed6ffa90 0x55d4ed6fc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.948+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 >> 172.21.15.82:0/2661337774 conn(0x55d4ed6ef3d0 msgr2=0x55d4ed700300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 shutdown_connections 2023-07-14T19:30:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f844a916b80 1 -- 172.21.15.82:0/2661337774 wait complete. 2023-07-14T19:30:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f844a916b80 20 client.25523 trim_cache size 0 max 16384 2023-07-14T19:30:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:30:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:30:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:30:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:30:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f84200a12c0 msgr2=0x7f84200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f84200a12c0 0x7f84200a3770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8428009c60 tx=0x7f8428009260 comp rx=0 tx=0).stop 2023-07-14T19:30:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 msgr2=0x7f84381984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 0x7f84381984a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f842c0098e0 tx=0x7f842c00f690 comp rx=0 tx=0).stop 2023-07-14T19:30:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 shutdown_connections 2023-07-14T19:30:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f84200a12c0 0x7f84200a3770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8438107110 0x7f843819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84381020d0 0x7f8438197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 --2- 172.21.15.82:0/3482561871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8438102a70 0x7f84381984a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 >> 172.21.15.82:0/3482561871 conn(0x7f84380fd960 msgr2=0x7f84380fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.949+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 shutdown_connections 2023-07-14T19:30:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:26.950+0000 7f843e77e700 1 -- 172.21.15.82:0/3482561871 wait complete. 2023-07-14T19:30:26.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:26 smithi082 ceph-mon[127461]: pgmap v60: 97 pgs: 97 active+clean; 268 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 957 op/s 2023-07-14T19:30:26.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:26 smithi161 ceph-mon[129184]: pgmap v60: 97 pgs: 97 active+clean; 268 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 957 op/s 2023-07-14T19:30:28.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:28 smithi133 ceph-mon[128894]: pgmap v61: 97 pgs: 97 active+clean; 268 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2023-07-14T19:30:28.973 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:28 smithi082 ceph-mon[127461]: pgmap v61: 97 pgs: 97 active+clean; 268 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2023-07-14T19:30:28.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:28 smithi161 ceph-mon[129184]: pgmap v61: 97 pgs: 97 active+clean; 268 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 158 op/s 2023-07-14T19:30:30.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:30 smithi133 ceph-mon[128894]: pgmap v62: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 515 op/s 2023-07-14T19:30:30.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:30 smithi082 ceph-mon[127461]: pgmap v62: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 515 op/s 2023-07-14T19:30:30.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:30 smithi161 ceph-mon[129184]: pgmap v62: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 515 op/s 2023-07-14T19:30:32.171 INFO:tasks.workunit.client.0.smithi082.stdout: 85 810780 1204 540518 1107 151078 997 2023-07-14T19:30:32.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:32 smithi133 ceph-mon[128894]: pgmap v63: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 462 op/s 2023-07-14T19:30:32.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:32 smithi082 ceph-mon[127461]: pgmap v63: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 462 op/s 2023-07-14T19:30:32.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:32 smithi161 ceph-mon[129184]: pgmap v63: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 462 op/s 2023-07-14T19:30:34.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:34 smithi133 ceph-mon[128894]: pgmap v64: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 462 op/s 2023-07-14T19:30:34.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:34 smithi082 ceph-mon[127461]: pgmap v64: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 462 op/s 2023-07-14T19:30:34.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:34 smithi161 ceph-mon[129184]: pgmap v64: 97 pgs: 97 active+clean; 273 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 462 op/s 2023-07-14T19:30:36.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:36 smithi133 ceph-mon[128894]: pgmap v65: 97 pgs: 97 active+clean; 283 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 546 op/s 2023-07-14T19:30:36.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:36 smithi082 ceph-mon[127461]: pgmap v65: 97 pgs: 97 active+clean; 283 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 546 op/s 2023-07-14T19:30:36.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:36 smithi161 ceph-mon[129184]: pgmap v65: 97 pgs: 97 active+clean; 283 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 546 op/s 2023-07-14T19:30:38.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:38 smithi082 ceph-mon[127461]: pgmap v66: 97 pgs: 97 active+clean; 283 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 440 op/s 2023-07-14T19:30:38.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:38 smithi161 ceph-mon[129184]: pgmap v66: 97 pgs: 97 active+clean; 283 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 440 op/s 2023-07-14T19:30:39.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:38 smithi133 ceph-mon[128894]: pgmap v66: 97 pgs: 97 active+clean; 283 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 440 op/s 2023-07-14T19:30:39.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:39 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:30:39.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:39 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:30:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 -- 172.21.15.82:0/3807499832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 msgr2=0x7f1fa810cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 --2- 172.21.15.82:0/3807499832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa810cb80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000b0a0 tx=0x7f1fa000b3b0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 -- 172.21.15.82:0/3807499832 shutdown_connections 2023-07-14T19:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 --2- 172.21.15.82:0/3807499832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa810cb80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 --2- 172.21.15.82:0/3807499832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa8067ca0 0x7f1fa8068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 --2- 172.21.15.82:0/3807499832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa80685b0 0x7f1fa8105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 -- 172.21.15.82:0/3807499832 >> 172.21.15.82:0/3807499832 conn(0x7f1fa8074cc0 msgr2=0x7f1fa80750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 -- 172.21.15.82:0/3807499832 shutdown_connections 2023-07-14T19:30:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.114+0000 7f1fb0565700 1 -- 172.21.15.82:0/3807499832 wait complete. 2023-07-14T19:30:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 Processor -- start 2023-07-14T19:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 -- start start 2023-07-14T19:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8067ca0 0x7f1fa8195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa80685b0 0x7f1fa8196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa819af40 con 0x7f1fa8106140 2023-07-14T19:30:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa819b0b0 con 0x7f1fa80685b0 2023-07-14T19:30:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fb0565700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa819b270 con 0x7f1fa8067ca0 2023-07-14T19:30:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fadb00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa80685b0 0x7f1fa8196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43696/0 (socket says 172.21.15.82:43696) 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1faeb02700 1 -- 172.21.15.82:0/2204730483 learned_addr learned my addr 172.21.15.82:0/2204730483 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.115+0000 7f1fae301700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8067ca0 0x7f1fa8195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fadb00700 1 -- 172.21.15.82:0/2204730483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8067ca0 msgr2=0x7f1fa8195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fadb00700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8067ca0 0x7f1fa8195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fadb00700 1 -- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 msgr2=0x7f1fa819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fadb00700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fadb00700 1 -- 172.21.15.82:0/2204730483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4009520 con 0x7f1fa80685b0 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fadb00700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa80685b0 0x7f1fa8196290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800e9f0 tx=0x7f1f9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1f9f7fe700 1 -- 172.21.15.82:0/2204730483 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800c9c0 con 0x7f1fa80685b0 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1f9f7fe700 1 -- 172.21.15.82:0/2204730483 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f98010430 con 0x7f1fa80685b0 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa0009d00 con 0x7f1fa80685b0 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1f9f7fe700 1 -- 172.21.15.82:0/2204730483 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98018680 con 0x7f1fa80685b0 2023-07-14T19:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.116+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa81a2bb0 con 0x7f1fa80685b0 2023-07-14T19:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.118+0000 7f1f9f7fe700 1 -- 172.21.15.82:0/2204730483 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1f98010860 con 0x7f1fa80685b0 2023-07-14T19:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.119+0000 7f1f9f7fe700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1f940a12c0 0x7f1f940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.119+0000 7f1f9f7fe700 1 -- 172.21.15.82:0/2204730483 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1f98014070 con 0x7f1fa80685b0 2023-07-14T19:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.119+0000 7f1fae301700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1f940a12c0 0x7f1f940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.120+0000 7f1fae301700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1f940a12c0 0x7f1f940a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009df0 tx=0x7f1fa4009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.120+0000 7f1fbc6fdb80 1 Processor -- start 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 -- start start 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d1e8f6b0 0x55f0d203d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 0x55f0d2037df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2038370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1f38c50 con 0x55f0d203e2d0 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1f8b9e0 con 0x55f0d203de60 2023-07-14T19:30:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1f84e70 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2038370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fadb00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 0x55f0d2037df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fae301700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d1e8f6b0 0x55f0d203d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fadb00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 0x55f0d2037df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40050/0 (socket says 172.21.15.82:40050) 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2038370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43704/0 (socket says 172.21.15.82:43704) 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fadb00700 1 -- 172.21.15.82:0/2922451075 learned_addr learned my addr 172.21.15.82:0/2922451075 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.121+0000 7f1fadb00700 1 -- 172.21.15.82:0/2922451075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d1e8f6b0 msgr2=0x55f0d203d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1fadb00700 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d1e8f6b0 0x55f0d203d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1fadb00700 1 -- 172.21.15.82:0/2922451075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 msgr2=0x55f0d2038370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1fadb00700 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2038370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1fadb00700 1 -- 172.21.15.82:0/2922451075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1db2d80 con 0x55f0d203de60 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1fae301700 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d1e8f6b0 0x55f0d203d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1fadb00700 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 0x55f0d2037df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f980a1e10 tx=0x7f1f98099df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f7499bba2682aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1f9cff9700 1 -- 172.21.15.82:0/2922451075 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f980a2340 con 0x55f0d203de60 2023-07-14T19:30:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1f9cff9700 1 -- 172.21.15.82:0/2922451075 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f980105a0 con 0x55f0d203de60 2023-07-14T19:30:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.122+0000 7f1f9cff9700 1 -- 172.21.15.82:0/2922451075 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f980a1eb0 con 0x55f0d203de60 2023-07-14T19:30:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/2922451075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 msgr2=0x55f0d2037df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 0x55f0d2037df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f980a1e10 tx=0x7f1f98099df0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/2922451075 shutdown_connections 2023-07-14T19:30:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2038370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d1e8f6b0 0x55f0d203d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/2922451075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203de60 0x55f0d2037df0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/2922451075 >> 172.21.15.82:0/2922451075 conn(0x55f0d20293d0 msgr2=0x55f0d202e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/2922451075 shutdown_connections 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/2922451075 wait complete. 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 -- start start 2023-07-14T19:30:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.123+0000 7f1fbc6fdb80 1 Processor -- start 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2044240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2041560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2041c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1f24a60 con 0x55f0d203e2d0 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1f24760 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1e088c0 con 0x55f0d203de60 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 10 client.0 _ll_register_callbacks cb 0x55f0d1f22700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2044240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2044240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40054/0 (socket says 172.21.15.82:40054) 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 -- 172.21.15.82:0/3941403367 learned_addr learned my addr 172.21.15.82:0/3941403367 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 -- 172.21.15.82:0/3941403367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 msgr2=0x55f0d2041560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2041560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 -- 172.21.15.82:0/3941403367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 msgr2=0x55f0d2041c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2041c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 -- 172.21.15.82:0/3941403367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4017070 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fae301700 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2044240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4003950 tx=0x7f1fa40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50805fd6f653aa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1f91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1f91ffb700 1 -- 172.21.15.82:0/3941403367 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4022070 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1f91ffb700 1 -- 172.21.15.82:0/3941403367 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1fa400eae0 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.124+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0d1ede490 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1f91ffb700 1 -- 172.21.15.82:0/3941403367 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fa4003e80 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 10 client.25535 fetch_fsmap learned FSMap version 30 2023-07-14T19:30:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1fa81a2bb0 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1f91ffb700 1 -- 172.21.15.82:0/3941403367 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0d1ede490 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1f91ffb700 1 -- 172.21.15.82:0/3941403367 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1fa401d410 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 10 client.25535 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 10 client.25535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 20 client.25535 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20518b0 0x55f0d2053cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 4 client.25535 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:30:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0d2002d40 con 0x55f0d20518b0 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.125+0000 7f1fadb00700 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20518b0 0x55f0d2053cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1f91ffb700 10 client.25535 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fadb00700 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20518b0 0x55f0d2053cc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e08e4d5bbe4730 server_cookie=cf876fce4b60628e in_seq=0 out_seq=0 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1f91ffb700 1 -- 172.21.15.82:0/3941403367 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55f0d2002d40 con 0x55f0d20518b0 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1f91ffb700 10 client.25535 handle_command_reply: tid=0 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 client.25535 shutdown 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 msgr2=0x55f0d2044240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2044240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4003950 tx=0x7f1fa40047c0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 shutdown_connections 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d2041c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2044240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20518b0 0x55f0d2053cc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/3941403367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2041560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1f91ffb700 0 client.25535 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.126+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 >> 172.21.15.82:0/3941403367 conn(0x55f0d20293d0 msgr2=0x55f0d202dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.127+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 shutdown_connections 2023-07-14T19:30:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.127+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/3941403367 wait complete. 2023-07-14T19:30:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.127+0000 7f1fbc6fdb80 20 client.25535 trim_cache size 0 max 16384 2023-07-14T19:30:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 Processor -- start 2023-07-14T19:30:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 -- start start 2023-07-14T19:30:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2048890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2043d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 0x55f0d2044240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d202f630 con 0x55f0d1e8f6b0 2023-07-14T19:30:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d202b450 con 0x55f0d203e2d0 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.154+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d203d5a0 con 0x55f0d203de60 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.155+0000 7f1fadb00700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2043d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.155+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 0x55f0d2044240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.155+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 0x55f0d2044240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40062/0 (socket says 172.21.15.82:40062) 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.155+0000 7f1faeb02700 1 -- 172.21.15.82:0/4150954122 learned_addr learned my addr 172.21.15.82:0/4150954122 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1faeb02700 1 -- 172.21.15.82:0/4150954122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 msgr2=0x55f0d2043d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1faeb02700 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2043d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1faeb02700 1 -- 172.21.15.82:0/4150954122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d1e8f6b0 msgr2=0x55f0d2048890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1faeb02700 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2048890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1faeb02700 1 -- 172.21.15.82:0/4150954122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1d9bc00 con 0x55f0d203e2d0 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1faeb02700 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 0x55f0d2044240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000b070 tx=0x7f1fa0009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=825fa568a14c9afb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1f9d7fa700 1 -- 172.21.15.82:0/4150954122 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000e040 con 0x55f0d203e2d0 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1f9d7fa700 1 -- 172.21.15.82:0/4150954122 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fa000bcb0 con 0x55f0d203e2d0 2023-07-14T19:30:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.156+0000 7f1f9d7fa700 1 -- 172.21.15.82:0/4150954122 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00052a0 con 0x55f0d203e2d0 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.157+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/4150954122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 msgr2=0x55f0d2044240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.157+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 0x55f0d2044240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000b070 tx=0x7f1fa0009410 comp rx=0 tx=0).stop 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.158+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/4150954122 shutdown_connections 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.158+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d203e2d0 0x55f0d2044240 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.158+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d1e8f6b0 0x55f0d2048890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.158+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/4150954122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2043d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.158+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/4150954122 >> 172.21.15.82:0/4150954122 conn(0x55f0d20293d0 msgr2=0x55f0d204cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:40.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:39 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:30:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.210+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/4150954122 shutdown_connections 2023-07-14T19:30:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.210+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/4150954122 wait complete. 2023-07-14T19:30:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 -- start start 2023-07-14T19:30:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 Processor -- start 2023-07-14T19:30:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2047460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d20479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.212 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 0x55f0d2036a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d20370e0 con 0x55f0d203e2d0 2023-07-14T19:30:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d2037250 con 0x55f0d2047f80 2023-07-14T19:30:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d2037390 con 0x55f0d203de60 2023-07-14T19:30:40.213 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.211+0000 7f1fbc6fdb80 10 client.0 _ll_register_callbacks cb 0x7f1fa4004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.212+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 0x55f0d2036a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.212+0000 7f1faeb02700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 0x55f0d2036a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40066/0 (socket says 172.21.15.82:40066) 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.212+0000 7f1faeb02700 1 -- 172.21.15.82:0/1558576393 learned_addr learned my addr 172.21.15.82:0/1558576393 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.212+0000 7f1fadb00700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d20479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.212+0000 7f1fae301700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2047460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.213+0000 7f1faeb02700 1 -- 172.21.15.82:0/1558576393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 msgr2=0x55f0d2047460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.213+0000 7f1faeb02700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2047460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.213+0000 7f1faeb02700 1 -- 172.21.15.82:0/1558576393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 msgr2=0x55f0d20479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.213+0000 7f1faeb02700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d20479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.213+0000 7f1faeb02700 1 -- 172.21.15.82:0/1558576393 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa810d340 con 0x55f0d2047f80 2023-07-14T19:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.237+0000 7f1faeb02700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 0x55f0d2036a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000b3f0 tx=0x7f1fa000bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daab1bdb44bad0bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.237+0000 7f1f91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.237+0000 7f1f91ffb700 1 -- 172.21.15.82:0/1558576393 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000e040 con 0x55f0d2047f80 2023-07-14T19:30:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.237+0000 7f1f91ffb700 1 -- 172.21.15.82:0/1558576393 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f1fa0005030 con 0x55f0d2047f80 2023-07-14T19:30:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.237+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f980105a0 con 0x55f0d2047f80 2023-07-14T19:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.263+0000 7f1f91ffb700 1 -- 172.21.15.82:0/1558576393 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fa00051f0 con 0x55f0d2047f80 2023-07-14T19:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.274+0000 7f1fbc6fdb80 10 client.25541 fetch_fsmap learned FSMap version 30 2023-07-14T19:30:40.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.274+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f0d1dd91f0 con 0x55f0d2047f80 2023-07-14T19:30:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.285+0000 7f1f91ffb700 1 -- 172.21.15.82:0/1558576393 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f980105a0 con 0x55f0d2047f80 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.285+0000 7f1f91ffb700 1 -- 172.21.15.82:0/1558576393 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1fa0045020 con 0x55f0d2047f80 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.289+0000 7f1fbc6fdb80 10 client.25541 fetch_fsmap finished waiting for FSMap version 30 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.289+0000 7f1fbc6fdb80 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.289+0000 7f1fbc6fdb80 20 client.25541 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.289+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20b32c0 0x55f0d20b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.289+0000 7f1fbc6fdb80 4 client.25541 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:30:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.289+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0d1f7de50 con 0x55f0d20b32c0 2023-07-14T19:30:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.299+0000 7f1fae301700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20b32c0 0x55f0d20b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.335+0000 7f1fae301700 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20b32c0 0x55f0d20b56d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbb263ae8a96f041 server_cookie=85efc402689e4527 in_seq=0 out_seq=0 2023-07-14T19:30:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.335+0000 7f1f91ffb700 10 client.25541 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.487+0000 7f1f91ffb700 1 -- 172.21.15.82:0/1558576393 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0d1f7de50 con 0x55f0d20b32c0 2023-07-14T19:30:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.487+0000 7f1f91ffb700 10 client.25541 handle_command_reply: tid=0 2023-07-14T19:30:40.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.487+0000 7f1fbc6fdb80 1 client.25541 shutdown 2023-07-14T19:30:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 msgr2=0x55f0d2036a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 0x55f0d2036a10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000b3f0 tx=0x7f1fa000bef0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 shutdown_connections 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f0d2047f80 0x55f0d2036a10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55f0d20b32c0 0x55f0d20b56d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f0d203de60 0x55f0d2047460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 --2- 172.21.15.82:0/1558576393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f0d203e2d0 0x55f0d20479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 >> 172.21.15.82:0/1558576393 conn(0x55f0d20293d0 msgr2=0x55f0d203b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1f91ffb700 0 client.25541 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.498+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 shutdown_connections 2023-07-14T19:30:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.499+0000 7f1fbc6fdb80 1 -- 172.21.15.82:0/1558576393 wait complete. 2023-07-14T19:30:40.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.499+0000 7f1fbc6fdb80 20 client.25541 trim_cache size 0 max 16384 2023-07-14T19:30:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:30:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:30:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:30:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:30:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1f940a12c0 msgr2=0x7f1f940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1f940a12c0 0x7f1f940a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009df0 tx=0x7f1fa4009260 comp rx=0 tx=0).stop 2023-07-14T19:30:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa80685b0 msgr2=0x7f1fa8196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa80685b0 0x7f1fa8196290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800e9f0 tx=0x7f1f9800ed00 comp rx=0 tx=0).stop 2023-07-14T19:30:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 shutdown_connections 2023-07-14T19:30:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1f940a12c0 0x7f1f940a3770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa8106140 0x7f1fa819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fa8067ca0 0x7f1fa8195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 --2- 172.21.15.82:0/2204730483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fa80685b0 0x7f1fa8196290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 >> 172.21.15.82:0/2204730483 conn(0x7f1fa8074cc0 msgr2=0x7f1fa8106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 shutdown_connections 2023-07-14T19:30:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:40.524+0000 7f1fb0565700 1 -- 172.21.15.82:0/2204730483 wait complete. 2023-07-14T19:30:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:40 smithi161 ceph-mon[129184]: pgmap v67: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 485 op/s 2023-07-14T19:30:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:40 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:30:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:40 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:30:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:40 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:30:41.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:40 smithi082 ceph-mon[127461]: pgmap v67: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 485 op/s 2023-07-14T19:30:41.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:40 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:30:41.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:40 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:30:41.012 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:40 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:30:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:40 smithi133 ceph-mon[128894]: pgmap v67: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 485 op/s 2023-07-14T19:30:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:40 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:30:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:40 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:30:41.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:40 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:30:42.171 INFO:tasks.workunit.client.0.smithi082.stdout: 106 1423248 1127 980722 1185 339082 769 2023-07-14T19:30:42.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:42 smithi161 ceph-mon[129184]: pgmap v68: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2023-07-14T19:30:43.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:42 smithi133 ceph-mon[128894]: pgmap v68: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2023-07-14T19:30:43.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:42 smithi082 ceph-mon[127461]: pgmap v68: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2023-07-14T19:30:44.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:44 smithi161 ceph-mon[129184]: pgmap v69: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2023-07-14T19:30:45.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:44 smithi133 ceph-mon[128894]: pgmap v69: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2023-07-14T19:30:45.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:44 smithi082 ceph-mon[127461]: pgmap v69: 97 pgs: 97 active+clean; 292 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 129 op/s 2023-07-14T19:30:46.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:46 smithi161 ceph-mon[129184]: pgmap v70: 97 pgs: 97 active+clean; 384 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 613 op/s 2023-07-14T19:30:47.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:46 smithi133 ceph-mon[128894]: pgmap v70: 97 pgs: 97 active+clean; 384 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 613 op/s 2023-07-14T19:30:47.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:46 smithi082 ceph-mon[127461]: pgmap v70: 97 pgs: 97 active+clean; 384 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 613 op/s 2023-07-14T19:30:49.186 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:48 smithi133 ceph-mon[128894]: pgmap v71: 97 pgs: 97 active+clean; 384 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.9 MiB/s wr, 529 op/s 2023-07-14T19:30:49.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:48 smithi082 ceph-mon[127461]: pgmap v71: 97 pgs: 97 active+clean; 384 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.9 MiB/s wr, 529 op/s 2023-07-14T19:30:49.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:48 smithi161 ceph-mon[129184]: pgmap v71: 97 pgs: 97 active+clean; 384 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.9 MiB/s wr, 529 op/s 2023-07-14T19:30:51.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:50 smithi133 ceph-mon[128894]: pgmap v72: 97 pgs: 97 active+clean; 402 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 610 op/s 2023-07-14T19:30:51.233 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:50 smithi082 ceph-mon[127461]: pgmap v72: 97 pgs: 97 active+clean; 402 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 610 op/s 2023-07-14T19:30:51.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:50 smithi161 ceph-mon[129184]: pgmap v72: 97 pgs: 97 active+clean; 402 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 610 op/s 2023-07-14T19:30:52.171 INFO:tasks.workunit.client.0.smithi082.stdout: 113 1210011 365 838514 256 319481 486 2023-07-14T19:30:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:52 smithi133 ceph-mon[128894]: pgmap v73: 97 pgs: 97 active+clean; 460 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 802 op/s 2023-07-14T19:30:53.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:52 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:52 smithi082 ceph-mon[127461]: pgmap v73: 97 pgs: 97 active+clean; 460 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 802 op/s 2023-07-14T19:30:53.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:52 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:53.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:52 smithi161 ceph-mon[129184]: pgmap v73: 97 pgs: 97 active+clean; 460 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 802 op/s 2023-07-14T19:30:53.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:52 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:30:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 -- 172.21.15.82:0/876120456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 msgr2=0x7f1958104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 --2- 172.21.15.82:0/876120456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f1958104900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f194c00fb20 tx=0x7f194c00fe30 comp rx=0 tx=0).stop 2023-07-14T19:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 -- 172.21.15.82:0/876120456 shutdown_connections 2023-07-14T19:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 --2- 172.21.15.82:0/876120456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1958104fe0 0x7f195810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 --2- 172.21.15.82:0/876120456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19580ffa10 0x7f19580ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 --2- 172.21.15.82:0/876120456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f1958104900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 -- 172.21.15.82:0/876120456 >> 172.21.15.82:0/876120456 conn(0x7f1958074cc0 msgr2=0x7f19580750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 -- 172.21.15.82:0/876120456 shutdown_connections 2023-07-14T19:30:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.700+0000 7f195cca8700 1 -- 172.21.15.82:0/876120456 wait complete. 2023-07-14T19:30:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 Processor -- start 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 -- start start 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19580ffa10 0x7f1958199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f195819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1958104fe0 0x7f19581939e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1958194050 con 0x7f19581003b0 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19581941c0 con 0x7f19580ffa10 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195cca8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1958194380 con 0x7f1958104fe0 2023-07-14T19:30:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f195819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f195819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43732/0 (socket says 172.21.15.82:43732) 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f1955d9b700 1 -- 172.21.15.82:0/1126717313 learned_addr learned my addr 172.21.15.82:0/1126717313 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f195659c700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19580ffa10 0x7f1958199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.701+0000 7f1956d9d700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1958104fe0 0x7f19581939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f1955d9b700 1 -- 172.21.15.82:0/1126717313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1958104fe0 msgr2=0x7f19581939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f1955d9b700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1958104fe0 0x7f19581939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f1955d9b700 1 -- 172.21.15.82:0/1126717313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19580ffa10 msgr2=0x7f1958199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f1955d9b700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19580ffa10 0x7f1958199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f1955d9b700 1 -- 172.21.15.82:0/1126717313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c00f750 con 0x7f19581003b0 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f1955d9b700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f195819a0c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f194c000c00 tx=0x7f194c00f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f19477fe700 1 -- 172.21.15.82:0/1126717313 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c003b60 con 0x7f19581003b0 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f19477fe700 1 -- 172.21.15.82:0/1126717313 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c004180 con 0x7f19581003b0 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1958194600 con 0x7f19581003b0 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f19477fe700 1 -- 172.21.15.82:0/1126717313 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c021460 con 0x7f19581003b0 2023-07-14T19:30:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.702+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19581a0b70 con 0x7f19581003b0 2023-07-14T19:30:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.704+0000 7f19477fe700 1 -- 172.21.15.82:0/1126717313 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f194c00e660 con 0x7f19581003b0 2023-07-14T19:30:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.705+0000 7f19477fe700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f19400a12c0 0x7f19400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.705+0000 7f19477fe700 1 -- 172.21.15.82:0/1126717313 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f194c0d2200 con 0x7f19581003b0 2023-07-14T19:30:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.705+0000 7f195659c700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f19400a12c0 0x7f19400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.705+0000 7f195659c700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f19400a12c0 0x7f19400a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1948009c60 tx=0x7f1948009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.706+0000 7f1968e40b80 1 Processor -- start 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.706+0000 7f1968e40b80 1 -- start start 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.706+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fba550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fb49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fb4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1968e40b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924eb9c50 con 0x55a924e106b0 2023-07-14T19:30:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1968e40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924f0c9e0 con 0x55a924fbaf00 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1968e40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924f05e70 con 0x55a924fbaa90 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f195659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fba550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f195659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fba550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43738/0 (socket says 172.21.15.82:43738) 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fb4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f195659c700 1 -- 172.21.15.82:0/1717200440 learned_addr learned my addr 172.21.15.82:0/1717200440 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1955d9b700 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fb49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 -- 172.21.15.82:0/1717200440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 msgr2=0x55a924fb49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fb49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 -- 172.21.15.82:0/1717200440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 msgr2=0x55a924fba550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fba550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 -- 172.21.15.82:0/1717200440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a924d33d80 con 0x55a924fbaf00 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f195659c700 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1955d9b700 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1956d9d700 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fb4f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f195000b380 tx=0x7f1950009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc261777e10b12b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.707+0000 7f1944ff9700 1 -- 172.21.15.82:0/1717200440 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950010040 con 0x55a924fbaf00 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1944ff9700 1 -- 172.21.15.82:0/1717200440 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1950004db0 con 0x55a924fbaf00 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1944ff9700 1 -- 172.21.15.82:0/1717200440 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950004890 con 0x55a924fbaf00 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 -- 172.21.15.82:0/1717200440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 msgr2=0x55a924fb4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fb4f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f195000b380 tx=0x7f1950009460 comp rx=0 tx=0).stop 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 -- 172.21.15.82:0/1717200440 shutdown_connections 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fb4f80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fba550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1717200440 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fb49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 -- 172.21.15.82:0/1717200440 >> 172.21.15.82:0/1717200440 conn(0x55a924faa3d0 msgr2=0x55a924faf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 -- 172.21.15.82:0/1717200440 shutdown_connections 2023-07-14T19:30:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.708+0000 7f1968e40b80 1 -- 172.21.15.82:0/1717200440 wait complete. 2023-07-14T19:30:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 -- start start 2023-07-14T19:30:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 Processor -- start 2023-07-14T19:30:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fc3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaa90 0x55a924fc4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaf00 0x55a924fbe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924ea5a60 con 0x55a924e106b0 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924ea5760 con 0x55a924fbaa90 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924d898c0 con 0x55a924fbaf00 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1968e40b80 10 client.0 _ll_register_callbacks cb 0x55a924ea3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f195659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fc3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f195659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fc3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43744/0 (socket says 172.21.15.82:43744) 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f195659c700 1 -- 172.21.15.82:0/4260709568 learned_addr learned my addr 172.21.15.82:0/4260709568 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1956d9d700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaf00 0x55a924fbe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.709+0000 7f1955d9b700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaa90 0x55a924fc4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f195659c700 1 -- 172.21.15.82:0/4260709568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaf00 msgr2=0x55a924fbe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f195659c700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaf00 0x55a924fbe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f195659c700 1 -- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaa90 msgr2=0x55a924fc4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f195659c700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaa90 0x55a924fc4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f195659c700 1 -- 172.21.15.82:0/4260709568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1948017070 con 0x55a924e106b0 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f1955d9b700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaa90 0x55a924fc4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f195659c700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fc3df0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1948005c50 tx=0x7f19480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db38b49a62d2b584 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f193dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f193dffb700 1 -- 172.21.15.82:0/4260709568 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948022070 con 0x55a924e106b0 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f193dffb700 1 -- 172.21.15.82:0/4260709568 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f194801d6e0 con 0x55a924e106b0 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a924e5f490 con 0x55a924e106b0 2023-07-14T19:30:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f193dffb700 1 -- 172.21.15.82:0/4260709568 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194801d8a0 con 0x55a924e106b0 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f1968e40b80 10 client.15867 fetch_fsmap learned FSMap version 31 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f19581a0b70 con 0x55a924e106b0 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f193dffb700 1 -- 172.21.15.82:0/4260709568 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a924e5f490 con 0x55a924e106b0 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.710+0000 7f193dffb700 1 -- 172.21.15.82:0/4260709568 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1948027a70 con 0x55a924e106b0 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1968e40b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1968e40b80 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1968e40b80 20 client.15867 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1968e40b80 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a924fd6b60 0x55a924fd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1968e40b80 4 client.15867 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a924f83d40 con 0x55a924fd6b60 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1955d9b700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a924fd6b60 0x55a924fd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f193dffb700 10 client.15867 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.711+0000 7f1955d9b700 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a924fd6b60 0x55a924fd8f70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eed732c3293d436 server_cookie=cd633af725ded390 in_seq=0 out_seq=0 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f193dffb700 1 -- 172.21.15.82:0/4260709568 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a924f83d40 con 0x55a924fd6b60 2023-07-14T19:30:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f193dffb700 10 client.15867 handle_command_reply: tid=0 2023-07-14T19:30:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 client.15867 shutdown 2023-07-14T19:30:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 msgr2=0x55a924fc3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fc3df0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1948005c50 tx=0x7f19480047c0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 shutdown_connections 2023-07-14T19:30:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a924fd6b60 0x55a924fd8f70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaf00 0x55a924fbe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f193dffb700 0 client.15867 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924e106b0 0x55a924fc3df0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 --2- 172.21.15.82:0/4260709568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaa90 0x55a924fc4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 >> 172.21.15.82:0/4260709568 conn(0x55a924faa3d0 msgr2=0x55a924faec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 shutdown_connections 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 1 -- 172.21.15.82:0/4260709568 wait complete. 2023-07-14T19:30:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.714+0000 7f1968e40b80 20 client.15867 trim_cache size 0 max 16384 2023-07-14T19:30:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 Processor -- start 2023-07-14T19:30:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 -- start start 2023-07-14T19:30:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 0x55a924fb9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fbe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbec00 0x55a924fbd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924fd4190 con 0x55a924fbaa90 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924fac450 con 0x55a924fbaf00 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1968e40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a924fbf620 con 0x55a924fbec00 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 0x55a924fb9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fbe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 0x55a924fb9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43752/0 (socket says 172.21.15.82:43752) 2023-07-14T19:30:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 -- 172.21.15.82:0/1901982341 learned_addr learned my addr 172.21.15.82:0/1901982341 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f1956d9d700 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbec00 0x55a924fbd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 -- 172.21.15.82:0/1901982341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbec00 msgr2=0x55a924fbd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbec00 0x55a924fbd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 -- 172.21.15.82:0/1901982341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 msgr2=0x55a924fbe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fbe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.731+0000 7f195659c700 1 -- 172.21.15.82:0/1901982341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195000b060 con 0x55a924fbaa90 2023-07-14T19:30:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f195659c700 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 0x55a924fb9a30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1948013010 tx=0x7f1948011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bda1479c584c7e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f19457fa700 1 -- 172.21.15.82:0/1901982341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1948022040 con 0x55a924fbaa90 2023-07-14T19:30:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f19457fa700 1 -- 172.21.15.82:0/1901982341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1948009ed0 con 0x55a924fbaa90 2023-07-14T19:30:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f19457fa700 1 -- 172.21.15.82:0/1901982341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194801d940 con 0x55a924fbaa90 2023-07-14T19:30:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 -- 172.21.15.82:0/1901982341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 msgr2=0x55a924fb9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 0x55a924fb9a30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1948013010 tx=0x7f1948011b30 comp rx=0 tx=0).stop 2023-07-14T19:30:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 -- 172.21.15.82:0/1901982341 shutdown_connections 2023-07-14T19:30:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbec00 0x55a924fbd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaa90 0x55a924fb9a30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 --2- 172.21.15.82:0/1901982341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a924fbaf00 0x55a924fbe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.732+0000 7f1968e40b80 1 -- 172.21.15.82:0/1901982341 >> 172.21.15.82:0/1901982341 conn(0x55a924faa3d0 msgr2=0x55a924fbc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 -- 172.21.15.82:0/1901982341 shutdown_connections 2023-07-14T19:30:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 -- 172.21.15.82:0/1901982341 wait complete. 2023-07-14T19:30:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 -- start start 2023-07-14T19:30:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 Processor -- start 2023-07-14T19:30:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fc2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 0x55a924fc26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9250277f0 0x55a925029bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a92502a150 con 0x55a924fbaf00 2023-07-14T19:30:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a92502a290 con 0x55a9250277f0 2023-07-14T19:30:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a92502a450 con 0x55a924fbaa90 2023-07-14T19:30:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1968e40b80 10 client.0 _ll_register_callbacks cb 0x7f19480044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:30:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 0x55a924fc26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 0x55a924fc26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43760/0 (socket says 172.21.15.82:43760) 2023-07-14T19:30:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1955d9b700 1 -- 172.21.15.82:0/673778561 learned_addr learned my addr 172.21.15.82:0/673778561 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:30:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f195659c700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fc2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.733+0000 7f1956d9d700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9250277f0 0x55a925029bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1955d9b700 1 -- 172.21.15.82:0/673778561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 msgr2=0x55a924fc2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.746 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1955d9b700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fc2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1955d9b700 1 -- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9250277f0 msgr2=0x55a925029bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:30:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1955d9b700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9250277f0 0x55a925029bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1955d9b700 1 -- 172.21.15.82:0/673778561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19581a0b70 con 0x55a924fbaf00 2023-07-14T19:30:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1955d9b700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 0x55a924fc26d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f194c00faf0 tx=0x7f194c09b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cea68edaf7532a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:30:53.747 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f193dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:30:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f193dffb700 1 -- 172.21.15.82:0/673778561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0d1040 con 0x55a924fbaf00 2023-07-14T19:30:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1950004db0 con 0x55a924fbaf00 2023-07-14T19:30:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f193dffb700 1 -- 172.21.15.82:0/673778561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f194c0042f0 con 0x55a924fbaf00 2023-07-14T19:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f193dffb700 1 -- 172.21.15.82:0/673778561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194c0bb020 con 0x55a924fbaf00 2023-07-14T19:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 10 client.15873 fetch_fsmap learned FSMap version 31 2023-07-14T19:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a924d1cc00 con 0x55a924fbaf00 2023-07-14T19:30:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f193dffb700 1 -- 172.21.15.82:0/673778561 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c0cd020 con 0x55a924fbaf00 2023-07-14T19:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f193dffb700 1 -- 172.21.15.82:0/673778561 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f194c0a50b0 con 0x55a924fbaf00 2023-07-14T19:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 31 2023-07-14T19:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:30:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 20 client.15873 populate_metadata read hostname 'smithi082' 2023-07-14T19:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a92502b8c0 0x55a92502dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 4 client.15873 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.734+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a924efee50 con 0x55a92502b8c0 2023-07-14T19:30:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.735+0000 7f195659c700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a92502b8c0 0x55a92502dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:30:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.735+0000 7f193dffb700 10 client.15873 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:53.735+0000 7f195659c700 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a92502b8c0 0x55a92502dcd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=352852aafff82d8 server_cookie=233695be9c686bfc in_seq=0 out_seq=0 2023-07-14T19:30:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.781+0000 7f193dffb700 1 -- 172.21.15.82:0/673778561 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a924efee50 con 0x55a92502b8c0 2023-07-14T19:30:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.781+0000 7f193dffb700 10 client.15873 handle_command_reply: tid=0 2023-07-14T19:30:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.781+0000 7f1968e40b80 1 client.15873 shutdown 2023-07-14T19:30:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 msgr2=0x55a924fc26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 0x55a924fc26d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f194c00faf0 tx=0x7f194c09b040 comp rx=0 tx=0).stop 2023-07-14T19:30:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 shutdown_connections 2023-07-14T19:30:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9250277f0 0x55a925029bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a92502b8c0 0x55a92502dcd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a924fbaa90 0x55a924fc2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 --2- 172.21.15.82:0/673778561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a924fbaf00 0x55a924fc26d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 >> 172.21.15.82:0/673778561 conn(0x55a924faa3d0 msgr2=0x55a924fb98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f193dffb700 0 client.15873 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:30:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 shutdown_connections 2023-07-14T19:30:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 1 -- 172.21.15.82:0/673778561 wait complete. 2023-07-14T19:30:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.783+0000 7f1968e40b80 20 client.15873 trim_cache size 0 max 16384 2023-07-14T19:30:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:30:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:30:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:30:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:30:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f19400a12c0 msgr2=0x7f19400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f19400a12c0 0x7f19400a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1948009c60 tx=0x7f1948009260 comp rx=0 tx=0).stop 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 msgr2=0x7f195819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f195819a0c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f194c000c00 tx=0x7f194c00f3e0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 shutdown_connections 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f19400a12c0 0x7f19400a3770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1958104fe0 0x7f19581939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19580ffa10 0x7f1958199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 --2- 172.21.15.82:0/1126717313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19581003b0 0x7f195819a0c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:30:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 >> 172.21.15.82:0/1126717313 conn(0x7f1958074cc0 msgr2=0x7f19581028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:30:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 shutdown_connections 2023-07-14T19:30:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:30:54.785+0000 7f195cca8700 1 -- 172.21.15.82:0/1126717313 wait complete. 2023-07-14T19:30:55.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:54 smithi133 ceph-mon[128894]: pgmap v74: 97 pgs: 97 active+clean; 460 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 802 op/s 2023-07-14T19:30:55.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:54 smithi082 ceph-mon[127461]: pgmap v74: 97 pgs: 97 active+clean; 460 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 802 op/s 2023-07-14T19:30:55.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:54 smithi161 ceph-mon[129184]: pgmap v74: 97 pgs: 97 active+clean; 460 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 15 MiB/s wr, 802 op/s 2023-07-14T19:30:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:56 smithi133 ceph-mon[128894]: pgmap v75: 97 pgs: 97 active+clean; 467 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.47k op/s 2023-07-14T19:30:57.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:56 smithi082 ceph-mon[127461]: pgmap v75: 97 pgs: 97 active+clean; 467 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.47k op/s 2023-07-14T19:30:57.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:56 smithi161 ceph-mon[129184]: pgmap v75: 97 pgs: 97 active+clean; 467 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.47k op/s 2023-07-14T19:30:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:30:58 smithi133 ceph-mon[128894]: pgmap v76: 97 pgs: 97 active+clean; 467 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 989 op/s 2023-07-14T19:30:59.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:30:58 smithi082 ceph-mon[127461]: pgmap v76: 97 pgs: 97 active+clean; 467 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 989 op/s 2023-07-14T19:30:59.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:30:58 smithi161 ceph-mon[129184]: pgmap v76: 97 pgs: 97 active+clean; 467 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 989 op/s 2023-07-14T19:31:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:00 smithi133 ceph-mon[128894]: pgmap v77: 97 pgs: 97 active+clean; 459 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 998 op/s 2023-07-14T19:31:01.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:00 smithi082 ceph-mon[127461]: pgmap v77: 97 pgs: 97 active+clean; 459 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 998 op/s 2023-07-14T19:31:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:00 smithi161 ceph-mon[129184]: pgmap v77: 97 pgs: 97 active+clean; 459 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 998 op/s 2023-07-14T19:31:02.172 INFO:tasks.workunit.client.0.smithi082.stdout: 123 1254945 578 874548 706 341130 597 2023-07-14T19:31:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:02 smithi133 ceph-mon[128894]: pgmap v78: 97 pgs: 97 active+clean; 464 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 957 op/s 2023-07-14T19:31:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:02 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:02 smithi082 ceph-mon[127461]: pgmap v78: 97 pgs: 97 active+clean; 464 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 957 op/s 2023-07-14T19:31:03.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:02 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:03.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:02 smithi161 ceph-mon[129184]: pgmap v78: 97 pgs: 97 active+clean; 464 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.5 MiB/s wr, 957 op/s 2023-07-14T19:31:03.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:02 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:05.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:04 smithi133 ceph-mon[128894]: pgmap v79: 97 pgs: 97 active+clean; 464 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2023-07-14T19:31:05.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:04 smithi082 ceph-mon[127461]: pgmap v79: 97 pgs: 97 active+clean; 464 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2023-07-14T19:31:05.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:04 smithi161 ceph-mon[129184]: pgmap v79: 97 pgs: 97 active+clean; 464 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 720 op/s 2023-07-14T19:31:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:06 smithi133 ceph-mon[128894]: pgmap v80: 97 pgs: 97 active+clean; 474 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 796 op/s 2023-07-14T19:31:07.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:06 smithi082 ceph-mon[127461]: pgmap v80: 97 pgs: 97 active+clean; 474 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 796 op/s 2023-07-14T19:31:07.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:06 smithi161 ceph-mon[129184]: pgmap v80: 97 pgs: 97 active+clean; 474 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 796 op/s 2023-07-14T19:31:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 -- 172.21.15.82:0/3598133302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 msgr2=0x7f11f00fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 --2- 172.21.15.82:0/3598133302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f00fef20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009990 tx=0x7f11e4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 -- 172.21.15.82:0/3598133302 shutdown_connections 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 --2- 172.21.15.82:0/3598133302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f00ff460 0x7f11f010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 --2- 172.21.15.82:0/3598133302 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f0104ac0 0x7f11f0104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 --2- 172.21.15.82:0/3598133302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f00fef20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 -- 172.21.15.82:0/3598133302 >> 172.21.15.82:0/3598133302 conn(0x7f11f00747b0 msgr2=0x7f11f0074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 -- 172.21.15.82:0/3598133302 shutdown_connections 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.906+0000 7f11f7933700 1 -- 172.21.15.82:0/3598133302 wait complete. 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 Processor -- start 2023-07-14T19:31:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 -- start start 2023-07-14T19:31:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f00ff460 0x7f11f01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f0104ac0 0x7f11f019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f019ce50 con 0x7f11f00feab0 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f019cfc0 con 0x7f11f00ff460 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f7933700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f019d180 con 0x7f11f0104ac0 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f0197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43772/0 (socket says 172.21.15.82:43772) 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 -- 172.21.15.82:0/3584475641 learned_addr learned my addr 172.21.15.82:0/3584475641 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f00ff460 0x7f11f01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f5ed0700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f0104ac0 0x7f11f019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 -- 172.21.15.82:0/3584475641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f0104ac0 msgr2=0x7f11f019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f0104ac0 0x7f11f019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 -- 172.21.15.82:0/3584475641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f00ff460 msgr2=0x7f11f01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.907+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f00ff460 0x7f11f01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11f56cf700 1 -- 172.21.15.82:0/3584475641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40095f0 con 0x7f11f00feab0 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f0197e70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00e970 tx=0x7f11ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11e27fc700 1 -- 172.21.15.82:0/3584475641 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec009790 con 0x7f11f00feab0 2023-07-14T19:31:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11e27fc700 1 -- 172.21.15.82:0/3584475641 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec00eec0 con 0x7f11f00feab0 2023-07-14T19:31:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f019d460 con 0x7f11f00feab0 2023-07-14T19:31:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11e27fc700 1 -- 172.21.15.82:0/3584475641 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec003cf0 con 0x7f11f00feab0 2023-07-14T19:31:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.908+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f01a4ca0 con 0x7f11f00feab0 2023-07-14T19:31:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.910+0000 7f11e27fc700 1 -- 172.21.15.82:0/3584475641 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f11ec00ccd0 con 0x7f11f00feab0 2023-07-14T19:31:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.910+0000 7f11e27fc700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f11dc0a12c0 0x7f11dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.911+0000 7f11e27fc700 1 -- 172.21.15.82:0/3584475641 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f11ec014070 con 0x7f11f00feab0 2023-07-14T19:31:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.911+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f11dc0a12c0 0x7f11dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.911+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f11dc0a12c0 0x7f11dc0a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11f0199480 tx=0x7f11e4005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.911+0000 7f1203acbb80 1 Processor -- start 2023-07-14T19:31:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 -- start start 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae7936b0 0x560cae93d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae937f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae83cc50 con 0x560cae93da90 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae88f9e0 con 0x560cae7936b0 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f1203acbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae888e70 con 0x560cae93df00 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43782/0 (socket says 172.21.15.82:43782) 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 -- 172.21.15.82:0/3952378088 learned_addr learned my addr 172.21.15.82:0/3952378088 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae7936b0 0x560cae93d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f5ed0700 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae937f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 -- 172.21.15.82:0/3952378088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 msgr2=0x560cae937f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae937f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 -- 172.21.15.82:0/3952378088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae7936b0 msgr2=0x560cae93d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae7936b0 0x560cae93d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 -- 172.21.15.82:0/3952378088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cae6b6d80 con 0x560cae93da90 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.912+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9379e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009990 tx=0x7f11e4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e796a8a889deb9e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f11db7fe700 1 -- 172.21.15.82:0/3952378088 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e402b070 con 0x560cae93da90 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f11db7fe700 1 -- 172.21.15.82:0/3952378088 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11e401e040 con 0x560cae93da90 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f11db7fe700 1 -- 172.21.15.82:0/3952378088 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400f6c0 con 0x560cae93da90 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 -- 172.21.15.82:0/3952378088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 msgr2=0x560cae9379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9379e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009990 tx=0x7f11e4020000 comp rx=0 tx=0).stop 2023-07-14T19:31:07.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 -- 172.21.15.82:0/3952378088 shutdown_connections 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae937f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae7936b0 0x560cae93d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3952378088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9379e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 -- 172.21.15.82:0/3952378088 >> 172.21.15.82:0/3952378088 conn(0x560cae92d3d0 msgr2=0x560cae932530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 -- 172.21.15.82:0/3952378088 shutdown_connections 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.913+0000 7f1203acbb80 1 -- 172.21.15.82:0/3952378088 wait complete. 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 -- start start 2023-07-14T19:31:07.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 Processor -- start 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae946cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93da90 0x560cae947210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae940f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae828a60 con 0x560cae7936b0 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae828760 con 0x560cae93df00 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae70c8c0 con 0x560cae93da90 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f1203acbb80 10 client.0 _ll_register_callbacks cb 0x560cae826700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:31:07.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae946cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae946cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43786/0 (socket says 172.21.15.82:43786) 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f11f56cf700 1 -- 172.21.15.82:0/3055398491 learned_addr learned my addr 172.21.15.82:0/3055398491 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93da90 0x560cae947210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.914+0000 7f11f5ed0700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae940f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11f56cf700 1 -- 172.21.15.82:0/3055398491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93da90 msgr2=0x560cae947210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93da90 0x560cae947210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11f56cf700 1 -- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 msgr2=0x560cae940f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae940f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11f56cf700 1 -- 172.21.15.82:0/3055398491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f010c1f0 con 0x560cae7936b0 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11f56cf700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae946cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f11ec099fd0 tx=0x7f11ec0163b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdabc448392d1cf server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.915+0000 7f11d8ff9700 1 -- 172.21.15.82:0/3055398491 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0a2490 con 0x560cae7936b0 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f11d8ff9700 1 -- 172.21.15.82:0/3055398491 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f11ec099530 con 0x560cae7936b0 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cae7e2490 con 0x560cae7936b0 2023-07-14T19:31:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f11d8ff9700 1 -- 172.21.15.82:0/3055398491 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec0996f0 con 0x560cae7936b0 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 10 client.15885 fetch_fsmap learned FSMap version 32 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560cae69fc00 con 0x560cae7936b0 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f11d8ff9700 1 -- 172.21.15.82:0/3055398491 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cae7e2490 con 0x560cae7936b0 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f11d8ff9700 1 -- 172.21.15.82:0/3055398491 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11ec0ca1a0 con 0x560cae7936b0 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 10 client.15885 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:31:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 20 client.15885 populate_metadata read hostname 'smithi082' 2023-07-14T19:31:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae951100 0x560cae9534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 4 client.15885 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:31:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.916+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cae906d40 con 0x560cae951100 2023-07-14T19:31:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.932+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae951100 0x560cae9534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.932+0000 7f11f4ece700 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae951100 0x560cae9534e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28214fbba56eab29 server_cookie=fc619132e184e6c2 in_seq=0 out_seq=0 2023-07-14T19:31:07.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.932+0000 7f11d8ff9700 10 client.15885 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f11d8ff9700 1 -- 172.21.15.82:0/3055398491 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560cae906d40 con 0x560cae951100 2023-07-14T19:31:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f11d8ff9700 10 client.15885 handle_command_reply: tid=0 2023-07-14T19:31:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 client.15885 shutdown 2023-07-14T19:31:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 msgr2=0x560cae946cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae946cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f11ec099fd0 tx=0x7f11ec0163b0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 shutdown_connections 2023-07-14T19:31:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae940f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae946cd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae951100 0x560cae9534e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 --2- 172.21.15.82:0/3055398491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93da90 0x560cae947210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f11d8ff9700 0 client.15885 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 >> 172.21.15.82:0/3055398491 conn(0x560cae92d3d0 msgr2=0x560cae931c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 shutdown_connections 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.933+0000 7f1203acbb80 1 -- 172.21.15.82:0/3055398491 wait complete. 2023-07-14T19:31:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.934+0000 7f1203acbb80 20 client.15885 trim_cache size 0 max 16384 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 Processor -- start 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 -- start start 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 0x560cae940f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae93e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae94e890 con 0x560cae93da90 2023-07-14T19:31:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae9337c0 con 0x560cae93df00 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f1203acbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae92f480 con 0x560cae7936b0 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f4ece700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 0x560cae940f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 0x560cae940f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53058/0 (socket says 172.21.15.82:53058) 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 -- 172.21.15.82:0/4111603958 learned_addr learned my addr 172.21.15.82:0/4111603958 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f5ed0700 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae93e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 -- 172.21.15.82:0/4111603958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 msgr2=0x560cae93e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae93e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 -- 172.21.15.82:0/4111603958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 msgr2=0x560cae9323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 -- 172.21.15.82:0/4111603958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cae6dd1f0 con 0x560cae7936b0 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f4ece700 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.950+0000 7f11f56cf700 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 0x560cae940f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0cacc0 tx=0x7f11ec003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585a3ffeb8d75df0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f11dbfff700 1 -- 172.21.15.82:0/4111603958 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0ca5d0 con 0x560cae7936b0 2023-07-14T19:31:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f11dbfff700 1 -- 172.21.15.82:0/4111603958 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec0a2490 con 0x560cae7936b0 2023-07-14T19:31:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f11dbfff700 1 -- 172.21.15.82:0/4111603958 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec099530 con 0x560cae7936b0 2023-07-14T19:31:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 -- 172.21.15.82:0/4111603958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 msgr2=0x560cae940f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 0x560cae940f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0cacc0 tx=0x7f11ec003780 comp rx=0 tx=0).stop 2023-07-14T19:31:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 -- 172.21.15.82:0/4111603958 shutdown_connections 2023-07-14T19:31:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93df00 0x560cae93e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae7936b0 0x560cae940f40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 --2- 172.21.15.82:0/4111603958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae93da90 0x560cae9323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 -- 172.21.15.82:0/4111603958 >> 172.21.15.82:0/4111603958 conn(0x560cae92d3d0 msgr2=0x560cae940830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 -- 172.21.15.82:0/4111603958 shutdown_connections 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.951+0000 7f1203acbb80 1 -- 172.21.15.82:0/4111603958 wait complete. 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 -- start start 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 Processor -- start 2023-07-14T19:31:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae944a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93da90 0x560cae944fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae9a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae9458b0 con 0x560cae7936b0 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae9a4280 con 0x560cae93da90 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cae9a43c0 con 0x560cae93df00 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f1203acbb80 10 client.0 _ll_register_callbacks cb 0x7f11ec099c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae944a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f11f56cf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae944a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43800/0 (socket says 172.21.15.82:43800) 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f11f56cf700 1 -- 172.21.15.82:0/2338360658 learned_addr learned my addr 172.21.15.82:0/2338360658 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f11f4ece700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93da90 0x560cae944fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.952+0000 7f11f5ed0700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae9a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11f56cf700 1 -- 172.21.15.82:0/2338360658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 msgr2=0x560cae9a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11f56cf700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae9a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11f56cf700 1 -- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93da90 msgr2=0x560cae944fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11f56cf700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93da90 0x560cae944fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11f56cf700 1 -- 172.21.15.82:0/2338360658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cae6b6d80 con 0x560cae7936b0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11f56cf700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae944a60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f11ec096540 tx=0x7f11ec099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ab4555e5ffc710 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11d8ff9700 1 -- 172.21.15.82:0/2338360658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0b8450 con 0x560cae7936b0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11d8ff9700 1 -- 172.21.15.82:0/2338360658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f11ec099530 con 0x560cae7936b0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11e401e040 con 0x560cae7936b0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11d8ff9700 1 -- 172.21.15.82:0/2338360658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11ec0ca5d0 con 0x560cae7936b0 2023-07-14T19:31:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 10 client.15891 fetch_fsmap learned FSMap version 32 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560cae69fc00 con 0x560cae7936b0 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11d8ff9700 1 -- 172.21.15.82:0/2338360658 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11ec0bd080 con 0x560cae7936b0 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f11d8ff9700 1 -- 172.21.15.82:0/2338360658 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11ec0b2a20 con 0x560cae7936b0 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 10 client.15891 fetch_fsmap finished waiting for FSMap version 32 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 20 client.15891 populate_metadata read hostname 'smithi082' 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.953+0000 7f1203acbb80 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae9a5580 0x560cae9a7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:07.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.954+0000 7f1203acbb80 4 client.15891 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:31:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.954+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560cae881e50 con 0x560cae9a5580 2023-07-14T19:31:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.970+0000 7f11f4ece700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae9a5580 0x560cae9a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.970+0000 7f11d8ff9700 10 client.15891 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:07.970+0000 7f11f4ece700 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae9a5580 0x560cae9a7960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93619fc78f327c8 server_cookie=3a5ce41b6794c19d in_seq=0 out_seq=0 2023-07-14T19:31:09.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:08 smithi133 ceph-mon[128894]: pgmap v81: 97 pgs: 97 active+clean; 474 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 124 op/s 2023-07-14T19:31:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:08 smithi082 ceph-mon[127461]: pgmap v81: 97 pgs: 97 active+clean; 474 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 124 op/s 2023-07-14T19:31:09.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:08 smithi161 ceph-mon[129184]: pgmap v81: 97 pgs: 97 active+clean; 474 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 124 op/s 2023-07-14T19:31:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:10 smithi133 ceph-mon[128894]: pgmap v82: 97 pgs: 97 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2023-07-14T19:31:11.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:10 smithi082 ceph-mon[127461]: pgmap v82: 97 pgs: 97 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2023-07-14T19:31:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:10 smithi161 ceph-mon[129184]: pgmap v82: 97 pgs: 97 active+clean; 479 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2023-07-14T19:31:12.172 INFO:tasks.workunit.client.0.smithi082.stdout: 176 130169 2400 91302 2650 34557 541 2023-07-14T19:31:13.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:12 smithi133 ceph-mon[128894]: pgmap v83: 97 pgs: 97 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 408 op/s 2023-07-14T19:31:13.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:12 smithi082 ceph-mon[127461]: pgmap v83: 97 pgs: 97 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 408 op/s 2023-07-14T19:31:13.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:12 smithi161 ceph-mon[129184]: pgmap v83: 97 pgs: 97 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 408 op/s 2023-07-14T19:31:15.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:14 smithi082 ceph-mon[127461]: pgmap v84: 97 pgs: 97 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 369 op/s 2023-07-14T19:31:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:14 smithi161 ceph-mon[129184]: pgmap v84: 97 pgs: 97 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 369 op/s 2023-07-14T19:31:15.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:14 smithi133 ceph-mon[128894]: pgmap v84: 97 pgs: 97 active+clean; 488 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 369 op/s 2023-07-14T19:31:17.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:16 smithi082 ceph-mon[127461]: pgmap v85: 97 pgs: 97 active+clean; 528 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 524 op/s 2023-07-14T19:31:17.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:16 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f11d8ff9700 1 -- 172.21.15.82:0/2338360658 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560cae881e50 con 0x560cae9a5580 2023-07-14T19:31:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f11d8ff9700 10 client.15891 handle_command_reply: tid=0 2023-07-14T19:31:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 client.15891 shutdown 2023-07-14T19:31:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 msgr2=0x560cae944a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae944a60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f11ec096540 tx=0x7f11ec099a50 comp rx=0 tx=0).stop 2023-07-14T19:31:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 shutdown_connections 2023-07-14T19:31:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560cae9a5580 0x560cae9a7960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560cae93df00 0x560cae9a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560cae7936b0 0x560cae944a60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 --2- 172.21.15.82:0/2338360658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cae93da90 0x560cae944fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 >> 172.21.15.82:0/2338360658 conn(0x560cae92d3d0 msgr2=0x560cae93e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f11d8ff9700 0 client.15891 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 shutdown_connections 2023-07-14T19:31:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 1 -- 172.21.15.82:0/2338360658 wait complete. 2023-07-14T19:31:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.222+0000 7f1203acbb80 20 client.15891 trim_cache size 0 max 16384 2023-07-14T19:31:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:31:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:31:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:31:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:31:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.223+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f11dc0a12c0 msgr2=0x7f11dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.223+0000 7f11f7933700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f11dc0a12c0 0x7f11dc0a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11f0199480 tx=0x7f11e4005fd0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.223+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 msgr2=0x7f11f0197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.223+0000 7f11f7933700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f0197e70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00e970 tx=0x7f11ec00ec80 comp rx=0 tx=0).stop 2023-07-14T19:31:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 shutdown_connections 2023-07-14T19:31:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f11dc0a12c0 0x7f11dc0a3770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11f0104ac0 0x7f11f019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f00feab0 0x7f11f0197e70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 --2- 172.21.15.82:0/3584475641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f00ff460 0x7f11f01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 >> 172.21.15.82:0/3584475641 conn(0x7f11f00747b0 msgr2=0x7f11f00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 shutdown_connections 2023-07-14T19:31:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:17.224+0000 7f11f7933700 1 -- 172.21.15.82:0/3584475641 wait complete. 2023-07-14T19:31:17.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:16 smithi161 ceph-mon[129184]: pgmap v85: 97 pgs: 97 active+clean; 528 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 524 op/s 2023-07-14T19:31:17.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:16 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:17.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:16 smithi133 ceph-mon[128894]: pgmap v85: 97 pgs: 97 active+clean; 528 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 524 op/s 2023-07-14T19:31:17.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:16 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:19.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:18 smithi161 ceph-mon[129184]: pgmap v86: 97 pgs: 97 active+clean; 528 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 448 op/s 2023-07-14T19:31:19.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:18 smithi133 ceph-mon[128894]: pgmap v86: 97 pgs: 97 active+clean; 528 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 448 op/s 2023-07-14T19:31:19.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:18 smithi082 ceph-mon[127461]: pgmap v86: 97 pgs: 97 active+clean; 528 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 448 op/s 2023-07-14T19:31:21.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:21 smithi133 ceph-mon[128894]: pgmap v87: 97 pgs: 97 active+clean; 564 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.4 MiB/s wr, 584 op/s 2023-07-14T19:31:21.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:21 smithi082 ceph-mon[127461]: pgmap v87: 97 pgs: 97 active+clean; 564 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.4 MiB/s wr, 584 op/s 2023-07-14T19:31:21.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:21 smithi161 ceph-mon[129184]: pgmap v87: 97 pgs: 97 active+clean; 564 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.4 MiB/s wr, 584 op/s 2023-07-14T19:31:22.172 INFO:tasks.workunit.client.0.smithi082.stdout: 184 903026 583 644382 480 208094 622 2023-07-14T19:31:22.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:22 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:22.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:22 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:22.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:22 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:23.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:23 smithi133 ceph-mon[128894]: pgmap v88: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 834 op/s 2023-07-14T19:31:23.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:23 smithi082 ceph-mon[127461]: pgmap v88: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 834 op/s 2023-07-14T19:31:23.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:23 smithi161 ceph-mon[129184]: pgmap v88: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 834 op/s 2023-07-14T19:31:25.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:25 smithi133 ceph-mon[128894]: pgmap v89: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 638 op/s 2023-07-14T19:31:25.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:25 smithi082 ceph-mon[127461]: pgmap v89: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 638 op/s 2023-07-14T19:31:25.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:25 smithi161 ceph-mon[129184]: pgmap v89: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 638 op/s 2023-07-14T19:31:27.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:27 smithi133 ceph-mon[128894]: pgmap v90: 97 pgs: 97 active+clean; 715 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 931 op/s 2023-07-14T19:31:27.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:27 smithi082 ceph-mon[127461]: pgmap v90: 97 pgs: 97 active+clean; 715 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 931 op/s 2023-07-14T19:31:27.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:27 smithi161 ceph-mon[129184]: pgmap v90: 97 pgs: 97 active+clean; 715 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 20 MiB/s wr, 931 op/s 2023-07-14T19:31:28.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:28 smithi133 ceph-mon[128894]: pgmap v91: 97 pgs: 97 active+clean; 736 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 897 op/s 2023-07-14T19:31:28.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:28 smithi082 ceph-mon[127461]: pgmap v91: 97 pgs: 97 active+clean; 736 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 897 op/s 2023-07-14T19:31:28.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:28 smithi161 ceph-mon[129184]: pgmap v91: 97 pgs: 97 active+clean; 736 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 18 MiB/s wr, 897 op/s 2023-07-14T19:31:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 -- 172.21.15.82:0/2469918349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 msgr2=0x7fcb2c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 --2- 172.21.15.82:0/2469918349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c072d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009990 tx=0x7fcb20009ca0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 -- 172.21.15.82:0/2469918349 shutdown_connections 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 --2- 172.21.15.82:0/2469918349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb2c10c540 0x7fcb2c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 --2- 172.21.15.82:0/2469918349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c072d50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 --2- 172.21.15.82:0/2469918349 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c073290 0x7fcb2c10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 -- 172.21.15.82:0/2469918349 >> 172.21.15.82:0/2469918349 conn(0x7fcb2c0fb820 msgr2=0x7fcb2c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 -- 172.21.15.82:0/2469918349 shutdown_connections 2023-07-14T19:31:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 -- 172.21.15.82:0/2469918349 wait complete. 2023-07-14T19:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 Processor -- start 2023-07-14T19:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.348+0000 7fcb33160700 1 -- start start 2023-07-14T19:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb33160700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb33160700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c073290 0x7fcb2c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb33160700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb2c10c540 0x7fcb2c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb33160700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19d110 con 0x7fcb2c072980 2023-07-14T19:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb33160700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19d280 con 0x7fcb2c10c540 2023-07-14T19:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb33160700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19d440 con 0x7fcb2c073290 2023-07-14T19:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43814/0 (socket says 172.21.15.82:43814) 2023-07-14T19:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 -- 172.21.15.82:0/2916148782 learned_addr learned my addr 172.21.15.82:0/2916148782 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c073290 0x7fcb2c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb316fd700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb2c10c540 0x7fcb2c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 -- 172.21.15.82:0/2916148782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c073290 msgr2=0x7fcb2c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c073290 0x7fcb2c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 -- 172.21.15.82:0/2916148782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb2c10c540 msgr2=0x7fcb2c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb2c10c540 0x7fcb2c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 -- 172.21.15.82:0/2916148782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200095f0 con 0x7fcb2c072980 2023-07-14T19:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c197f20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c00 tx=0x7fcb20004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.349+0000 7fcb29ffb700 1 -- 172.21.15.82:0/2916148782 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2001d070 con 0x7fcb2c072980 2023-07-14T19:31:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.350+0000 7fcb29ffb700 1 -- 172.21.15.82:0/2916148782 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb20021410 con 0x7fcb2c072980 2023-07-14T19:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.350+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c19d6c0 con 0x7fcb2c072980 2023-07-14T19:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.350+0000 7fcb29ffb700 1 -- 172.21.15.82:0/2916148782 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20017650 con 0x7fcb2c072980 2023-07-14T19:31:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.350+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1a4f60 con 0x7fcb2c072980 2023-07-14T19:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.352+0000 7fcb29ffb700 1 -- 172.21.15.82:0/2916148782 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcb20021580 con 0x7fcb2c072980 2023-07-14T19:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.352+0000 7fcb29ffb700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fcb1c0a12c0 0x7fcb1c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb29ffb700 1 -- 172.21.15.82:0/2916148782 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fcb200cdf10 con 0x7fcb2c072980 2023-07-14T19:31:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fcb1c0a12c0 0x7fcb1c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fcb1c0a12c0 0x7fcb1c0a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c199740 tx=0x7fcb18009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb3f2f8b80 1 Processor -- start 2023-07-14T19:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb3f2f8b80 1 -- start start 2023-07-14T19:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 0x5622478e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.353+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622478e5a90 0x5622478df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622477e4c50 con 0x5622478e5a90 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622478379e0 con 0x56224773b6b0 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247830e70 con 0x5622478e5f00 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 0x5622478e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622478e5a90 0x5622478df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 0x5622478e5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40170/0 (socket says 172.21.15.82:40170) 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 -- 172.21.15.82:0/2820865518 learned_addr learned my addr 172.21.15.82:0/2820865518 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 -- 172.21.15.82:0/2820865518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 msgr2=0x5622478dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 -- 172.21.15.82:0/2820865518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622478e5a90 msgr2=0x5622478df9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622478e5a90 0x5622478df9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 -- 172.21.15.82:0/2820865518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56224765ed80 con 0x56224773b6b0 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 0x5622478e5550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb2009b090 tx=0x7fcb2009afc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a09063784e0090 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb16ffd700 1 -- 172.21.15.82:0/2820865518 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2009b100 con 0x56224773b6b0 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb16ffd700 1 -- 172.21.15.82:0/2820865518 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2009f130 con 0x56224773b6b0 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.354+0000 7fcb16ffd700 1 -- 172.21.15.82:0/2820865518 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2009a830 con 0x56224773b6b0 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2820865518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 msgr2=0x5622478e5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 0x5622478e5550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcb2009b090 tx=0x7fcb2009afc0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2820865518 shutdown_connections 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56224773b6b0 0x5622478e5550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2820865518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622478e5a90 0x5622478df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2820865518 >> 172.21.15.82:0/2820865518 conn(0x5622478d53d0 msgr2=0x5622478da530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2820865518 shutdown_connections 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.355+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2820865518 wait complete. 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 -- start start 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 Processor -- start 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478eea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622477d0a60 con 0x56224773b6b0 2023-07-14T19:31:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622477d0760 con 0x5622478e5a90 2023-07-14T19:31:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622476b48c0 con 0x5622478e5f00 2023-07-14T19:31:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.356+0000 7fcb3f2f8b80 10 client.0 _ll_register_callbacks cb 0x5622477ce700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478eef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40178/0 (socket says 172.21.15.82:40178) 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478eea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43828/0 (socket says 172.21.15.82:43828) 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 -- 172.21.15.82:0/2857083836 learned_addr learned my addr 172.21.15.82:0/2857083836 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb316fd700 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 -- 172.21.15.82:0/2857083836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 msgr2=0x5622478e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 -- 172.21.15.82:0/2857083836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 msgr2=0x5622478eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 -- 172.21.15.82:0/2857083836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1a4f60 con 0x5622478e5a90 2023-07-14T19:31:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478eef70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb180045d0 tx=0x7fcb180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63b1bf4ec6ad4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb0bfff700 1 -- 172.21.15.82:0/2857083836 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18022070 con 0x5622478e5a90 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb0bfff700 1 -- 172.21.15.82:0/2857083836 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fcb18003e20 con 0x5622478e5a90 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56224778a490 con 0x5622478e5a90 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb0bfff700 1 -- 172.21.15.82:0/2857083836 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb1801d620 con 0x5622478e5a90 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb3f2f8b80 10 client.25589 fetch_fsmap learned FSMap version 34 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.357+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562247647c00 con 0x5622478e5a90 2023-07-14T19:31:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb0bfff700 1 -- 172.21.15.82:0/2857083836 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56224778a490 con 0x5622478e5a90 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb0bfff700 1 -- 172.21.15.82:0/2857083836 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcb18027890 con 0x5622478e5a90 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb3f2f8b80 10 client.25589 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb3f2f8b80 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb3f2f8b80 20 client.25589 populate_metadata read hostname 'smithi082' 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5622478f8fe0 0x5622478fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb3f2f8b80 4 client.25589 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622478aed40 con 0x5622478f8fe0 2023-07-14T19:31:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5622478f8fe0 0x5622478fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb0bfff700 10 client.25589 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.358+0000 7fcb30efc700 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5622478f8fe0 0x5622478fb3f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eea7cc46da20642 server_cookie=24c8ba62d966f100 in_seq=0 out_seq=0 2023-07-14T19:31:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.359+0000 7fcb0bfff700 1 -- 172.21.15.82:0/2857083836 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5622478aed40 con 0x5622478f8fe0 2023-07-14T19:31:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.359+0000 7fcb0bfff700 10 client.25589 handle_command_reply: tid=0 2023-07-14T19:31:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.359+0000 7fcb3f2f8b80 1 client.25589 shutdown 2023-07-14T19:31:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 msgr2=0x5622478eef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478eef70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb180045d0 tx=0x7fcb180047c0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 shutdown_connections 2023-07-14T19:31:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5622478f8fe0 0x5622478fb3f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb0bfff700 0 client.25589 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/2857083836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478eef70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 >> 172.21.15.82:0/2857083836 conn(0x5622478d53d0 msgr2=0x5622478d9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 shutdown_connections 2023-07-14T19:31:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/2857083836 wait complete. 2023-07-14T19:31:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.360+0000 7fcb3f2f8b80 20 client.25589 trim_cache size 0 max 16384 2023-07-14T19:31:30.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.375+0000 7fcb3f2f8b80 1 Processor -- start 2023-07-14T19:31:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.375+0000 7fcb3f2f8b80 1 -- start start 2023-07-14T19:31:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622478e9e30 con 0x56224773b6b0 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622478aed70 con 0x5622478e5a90 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622478dff20 con 0x5622478e5f00 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478df9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43836/0 (socket says 172.21.15.82:43836) 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb316fd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 -- 172.21.15.82:0/1944266284 learned_addr learned my addr 172.21.15.82:0/1944266284 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 -- 172.21.15.82:0/1944266284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 msgr2=0x5622478da3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478da3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 -- 172.21.15.82:0/1944266284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 msgr2=0x5622478e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478e8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 -- 172.21.15.82:0/1944266284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622476851f0 con 0x56224773b6b0 2023-07-14T19:31:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb30efc700 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478df9e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcb200059d0 tx=0x7fcb20009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380aaa33282e4a2c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb177fe700 1 -- 172.21.15.82:0/1944266284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2009b100 con 0x56224773b6b0 2023-07-14T19:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb177fe700 1 -- 172.21.15.82:0/1944266284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb200a34c0 con 0x56224773b6b0 2023-07-14T19:31:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.376+0000 7fcb177fe700 1 -- 172.21.15.82:0/1944266284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2009b360 con 0x56224773b6b0 2023-07-14T19:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1944266284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 msgr2=0x5622478df9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478df9e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fcb200059d0 tx=0x7fcb20009ca0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1944266284 shutdown_connections 2023-07-14T19:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x5622478da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x5622478df9e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1944266284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x5622478e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1944266284 >> 172.21.15.82:0/1944266284 conn(0x5622478d53d0 msgr2=0x5622478e8560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1944266284 shutdown_connections 2023-07-14T19:31:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1944266284 wait complete. 2023-07-14T19:31:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.377+0000 7fcb3f2f8b80 1 -- start start 2023-07-14T19:31:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 Processor -- start 2023-07-14T19:31:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x562247940bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x562247941110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x56224794bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562247941a20 con 0x56224773b6b0 2023-07-14T19:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224794c190 con 0x5622478e5a90 2023-07-14T19:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224794c2d0 con 0x5622478e5f00 2023-07-14T19:31:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb3f2f8b80 10 client.0 _ll_register_callbacks cb 0x7fcb180043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x562247940bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x562247941110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb30efc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x562247940bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43842/0 (socket says 172.21.15.82:43842) 2023-07-14T19:31:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb30efc700 1 -- 172.21.15.82:0/1477348517 learned_addr learned my addr 172.21.15.82:0/1477348517 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb316fd700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x56224794bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb2bfff700 1 -- 172.21.15.82:0/1477348517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 msgr2=0x56224794bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x56224794bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb2bfff700 1 -- 172.21.15.82:0/1477348517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 msgr2=0x562247940bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x562247940bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb2bfff700 1 -- 172.21.15.82:0/1477348517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56224765ed80 con 0x5622478e5a90 2023-07-14T19:31:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.378+0000 7fcb30efc700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x562247940bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:31:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb2bfff700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x562247941110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb18027c10 tx=0x7fcb18011a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af83ec8b3e304267 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:31:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb0bfff700 1 -- 172.21.15.82:0/1477348517 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18022040 con 0x5622478e5a90 2023-07-14T19:31:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb2009f130 con 0x5622478e5a90 2023-07-14T19:31:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb0bfff700 1 -- 172.21.15.82:0/1477348517 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fcb18027630 con 0x5622478e5a90 2023-07-14T19:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb0bfff700 1 -- 172.21.15.82:0/1477348517 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb180277f0 con 0x5622478e5a90 2023-07-14T19:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 10 client.25595 fetch_fsmap learned FSMap version 34 2023-07-14T19:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56224765d250 con 0x5622478e5a90 2023-07-14T19:31:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb0bfff700 1 -- 172.21.15.82:0/1477348517 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2009f130 con 0x5622478e5a90 2023-07-14T19:31:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb0bfff700 1 -- 172.21.15.82:0/1477348517 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcb1801c6a0 con 0x5622478e5a90 2023-07-14T19:31:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 10 client.25595 fetch_fsmap finished waiting for FSMap version 34 2023-07-14T19:31:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 10 client.25595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 20 client.25595 populate_metadata read hostname 'smithi082' 2023-07-14T19:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.379+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x56224794da90 0x56224794fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.380+0000 7fcb3f2f8b80 4 client.25595 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:31:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.380+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562247829e50 con 0x56224794da90 2023-07-14T19:31:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.380+0000 7fcb30efc700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x56224794da90 0x56224794fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.380+0000 7fcb0bfff700 10 client.25595 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:30.380+0000 7fcb30efc700 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x56224794da90 0x56224794fea0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492aa66df97fb78 server_cookie=f44cacc15a94c129 in_seq=0 out_seq=0 2023-07-14T19:31:30.935 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:30 smithi133 ceph-mon[128894]: pgmap v92: 97 pgs: 97 active+clean; 736 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2023-07-14T19:31:30.965 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:30 smithi082 ceph-mon[127461]: pgmap v92: 97 pgs: 97 active+clean; 736 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2023-07-14T19:31:30.993 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:30 smithi161 ceph-mon[129184]: pgmap v92: 97 pgs: 97 active+clean; 736 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2023-07-14T19:31:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb0bfff700 1 -- 172.21.15.82:0/1477348517 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562247829e50 con 0x56224794da90 2023-07-14T19:31:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb0bfff700 10 client.25595 handle_command_reply: tid=0 2023-07-14T19:31:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 client.25595 shutdown 2023-07-14T19:31:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 msgr2=0x562247941110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x562247941110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb18027c10 tx=0x7fcb18011a40 comp rx=0 tx=0).stop 2023-07-14T19:31:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 shutdown_connections 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x56224794da90 0x56224794fea0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5622478e5f00 0x56224794bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56224773b6b0 0x562247940bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 --2- 172.21.15.82:0/1477348517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622478e5a90 0x562247941110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 >> 172.21.15.82:0/1477348517 conn(0x5622478d53d0 msgr2=0x5622478df260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb0bfff700 0 client.25595 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 shutdown_connections 2023-07-14T19:31:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 1 -- 172.21.15.82:0/1477348517 wait complete. 2023-07-14T19:31:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.421+0000 7fcb3f2f8b80 20 client.25595 trim_cache size 0 max 16384 2023-07-14T19:31:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:31:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:31:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:31:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:31:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fcb1c0a12c0 msgr2=0x7fcb1c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fcb1c0a12c0 0x7fcb1c0a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c199740 tx=0x7fcb18009260 comp rx=0 tx=0).stop 2023-07-14T19:31:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 msgr2=0x7fcb2c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c197f20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb20000c00 tx=0x7fcb20004790 comp rx=0 tx=0).stop 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 shutdown_connections 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fcb1c0a12c0 0x7fcb1c0a3770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb2c10c540 0x7fcb2c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb2c072980 0x7fcb2c197f20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 --2- 172.21.15.82:0/2916148782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c073290 0x7fcb2c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 >> 172.21.15.82:0/2916148782 conn(0x7fcb2c0fb820 msgr2=0x7fcb2c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 shutdown_connections 2023-07-14T19:31:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:31.422+0000 7fcb33160700 1 -- 172.21.15.82:0/2916148782 wait complete. 2023-07-14T19:31:32.172 INFO:tasks.workunit.client.0.smithi082.stdout: 200 1313681 775 932486 622 319259 1089 2023-07-14T19:31:32.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:32 smithi133 ceph-mon[128894]: pgmap v93: 97 pgs: 97 active+clean; 743 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-07-14T19:31:32.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:32 smithi082 ceph-mon[127461]: pgmap v93: 97 pgs: 97 active+clean; 743 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-07-14T19:31:32.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:32 smithi161 ceph-mon[129184]: pgmap v93: 97 pgs: 97 active+clean; 743 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 16 MiB/s wr, 1.04k op/s 2023-07-14T19:31:34.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:34 smithi133 ceph-mon[128894]: pgmap v94: 97 pgs: 97 active+clean; 743 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.4 MiB/s wr, 691 op/s 2023-07-14T19:31:34.963 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:34 smithi082 ceph-mon[127461]: pgmap v94: 97 pgs: 97 active+clean; 743 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.4 MiB/s wr, 691 op/s 2023-07-14T19:31:34.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:34 smithi161 ceph-mon[129184]: pgmap v94: 97 pgs: 97 active+clean; 743 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.4 MiB/s wr, 691 op/s 2023-07-14T19:31:36.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:36 smithi133 ceph-mon[128894]: pgmap v95: 97 pgs: 97 active+clean; 738 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2023-07-14T19:31:36.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:36 smithi082 ceph-mon[127461]: pgmap v95: 97 pgs: 97 active+clean; 738 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2023-07-14T19:31:36.990 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:36 smithi161 ceph-mon[129184]: pgmap v95: 97 pgs: 97 active+clean; 738 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2023-07-14T19:31:38.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:38 smithi082 ceph-mon[127461]: pgmap v96: 97 pgs: 97 active+clean; 731 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 896 op/s 2023-07-14T19:31:38.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:38 smithi161 ceph-mon[129184]: pgmap v96: 97 pgs: 97 active+clean; 731 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 896 op/s 2023-07-14T19:31:39.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:38 smithi133 ceph-mon[128894]: pgmap v96: 97 pgs: 97 active+clean; 731 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 896 op/s 2023-07-14T19:31:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:40 smithi161 ceph-mon[129184]: pgmap v97: 97 pgs: 97 active+clean; 733 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 785 op/s 2023-07-14T19:31:40.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:40 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:31:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:40 smithi133 ceph-mon[128894]: pgmap v97: 97 pgs: 97 active+clean; 733 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 785 op/s 2023-07-14T19:31:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:40 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:31:41.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:40 smithi082 ceph-mon[127461]: pgmap v97: 97 pgs: 97 active+clean; 733 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 785 op/s 2023-07-14T19:31:41.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:40 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:31:42.180 INFO:tasks.workunit.client.0.smithi082.stdout: 203 1730659 359 1225081 288 464880 1232 2023-07-14T19:31:42.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:42 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:42 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:42 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:42 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:42 smithi133 ceph-mon[128894]: pgmap v98: 97 pgs: 97 active+clean; 735 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 712 op/s 2023-07-14T19:31:42.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:42 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:42 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:42 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:42 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:42 smithi082 ceph-mon[127461]: pgmap v98: 97 pgs: 97 active+clean; 735 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 712 op/s 2023-07-14T19:31:42.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:42 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:42 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:42 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:42 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:42.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:42 smithi161 ceph-mon[129184]: pgmap v98: 97 pgs: 97 active+clean; 735 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 712 op/s 2023-07-14T19:31:44.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:44 smithi082 ceph-mon[127461]: pgmap v99: 97 pgs: 97 active+clean; 735 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 547 op/s 2023-07-14T19:31:44.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:44 smithi161 ceph-mon[129184]: pgmap v99: 97 pgs: 97 active+clean; 735 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 547 op/s 2023-07-14T19:31:45.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:44 smithi133 ceph-mon[128894]: pgmap v99: 97 pgs: 97 active+clean; 735 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 547 op/s 2023-07-14T19:31:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.586+0000 7ffaca6a5700 1 -- 172.21.15.82:0/2128553873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406e640 msgr2=0x7ffac411b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.586+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/2128553873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406e640 0x7ffac411b5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009f90 tx=0x7ffabc00b330 comp rx=0 tx=0).stop 2023-07-14T19:31:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 -- 172.21.15.82:0/2128553873 shutdown_connections 2023-07-14T19:31:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/2128553873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406e640 0x7ffac411b5a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/2128553873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffac410e1a0 0x7ffac410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/2128553873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac406dc90 0x7ffac406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 -- 172.21.15.82:0/2128553873 >> 172.21.15.82:0/2128553873 conn(0x7ffac4071000 msgr2=0x7ffac4071400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 -- 172.21.15.82:0/2128553873 shutdown_connections 2023-07-14T19:31:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.669+0000 7ffaca6a5700 1 -- 172.21.15.82:0/2128553873 wait complete. 2023-07-14T19:31:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.670+0000 7ffaca6a5700 1 Processor -- start 2023-07-14T19:31:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.670+0000 7ffaca6a5700 1 -- start start 2023-07-14T19:31:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.670+0000 7ffaca6a5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 0x7ffac41aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.670+0000 7ffaca6a5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac406e640 0x7ffac41aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.672 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.671+0000 7ffaca6a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffac410e1a0 0x7ffac41a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.671+0000 7ffaca6a5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41a5030 con 0x7ffac410e1a0 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.671+0000 7ffaca6a5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41a51a0 con 0x7ffac406dc90 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.671+0000 7ffaca6a5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac41a5360 con 0x7ffac406e640 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac8ea2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac406e640 0x7ffac41aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 0x7ffac41aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 0x7ffac41aa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40206/0 (socket says 172.21.15.82:40206) 2023-07-14T19:31:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 -- 172.21.15.82:0/1467957558 learned_addr learned my addr 172.21.15.82:0/1467957558 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 -- 172.21.15.82:0/1467957558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac406e640 msgr2=0x7ffac41aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac406e640 0x7ffac41aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 -- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffac410e1a0 msgr2=0x7ffac41a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffac410e1a0 0x7ffac41a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.672+0000 7ffac96a3700 1 -- 172.21.15.82:0/1467957558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc009c40 con 0x7ffac406dc90 2023-07-14T19:31:45.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.681+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 0x7ffac41aa960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffac000d980 tx=0x7ffac000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.729+0000 7ffaba7fc700 1 -- 172.21.15.82:0/1467957558 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0009930 con 0x7ffac406dc90 2023-07-14T19:31:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.729+0000 7ffaba7fc700 1 -- 172.21.15.82:0/1467957558 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac0010460 con 0x7ffac406dc90 2023-07-14T19:31:45.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.729+0000 7ffaba7fc700 1 -- 172.21.15.82:0/1467957558 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000f620 con 0x7ffac406dc90 2023-07-14T19:31:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.730+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac41a5640 con 0x7ffac406dc90 2023-07-14T19:31:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.730+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac410ef50 con 0x7ffac406dc90 2023-07-14T19:31:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.736+0000 7ffad7a9fb80 1 Processor -- start 2023-07-14T19:31:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 -- start start 2023-07-14T19:31:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 0x556bdeb1b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb15c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdea1ac50 con 0x556bdeb1bcc0 2023-07-14T19:31:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdea6d9e0 con 0x556bdeb16170 2023-07-14T19:31:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdea66e70 con 0x556bde9716b0 2023-07-14T19:31:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.737+0000 7ffac8ea2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.738+0000 7ffac9ea4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 0x556bdeb1b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.738+0000 7ffac9ea4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 0x556bdeb1b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53128/0 (socket says 172.21.15.82:53128) 2023-07-14T19:31:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.738+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1483189659 learned_addr learned my addr 172.21.15.82:0/1483189659 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.741+0000 7ffaba7fc700 1 -- 172.21.15.82:0/1467957558 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ffac00105d0 con 0x7ffac406dc90 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.742+0000 7ffaba7fc700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ffab00a14d0 0x7ffab00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.742+0000 7ffaba7fc700 1 -- 172.21.15.82:0/1467957558 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7ffac00cc930 con 0x7ffac406dc90 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.747+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb15c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.748+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1483189659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 msgr2=0x556bdeb165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.748+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.748+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1483189659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 msgr2=0x556bdeb15c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.748+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb15c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.748+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1483189659 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bde894d80 con 0x556bde9716b0 2023-07-14T19:31:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.749+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 0x556bdeb1b780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009790 tx=0x7ffabc0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a43005680a0d75 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:45.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.759+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ffab00a14d0 0x7ffab00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.767+0000 7ffaaf7fe700 1 -- 172.21.15.82:0/1483189659 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00e040 con 0x556bde9716b0 2023-07-14T19:31:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.767+0000 7ffaaf7fe700 1 -- 172.21.15.82:0/1483189659 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffabc008ef0 con 0x556bde9716b0 2023-07-14T19:31:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.767+0000 7ffaaf7fe700 1 -- 172.21.15.82:0/1483189659 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc01d470 con 0x556bde9716b0 2023-07-14T19:31:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.768+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1483189659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 msgr2=0x556bdeb1b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.768+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 0x556bdeb1b780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009790 tx=0x7ffabc0093e0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.771+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ffab00a14d0 0x7ffab00a3980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffabc021000 tx=0x7ffabc01d780 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1483189659 shutdown_connections 2023-07-14T19:31:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bde9716b0 0x556bdeb1b780 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1483189659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb15c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1483189659 >> 172.21.15.82:0/1483189659 conn(0x556bdeb0b620 msgr2=0x556bdeb10660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1483189659 shutdown_connections 2023-07-14T19:31:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.779+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1483189659 wait complete. 2023-07-14T19:31:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.780+0000 7ffad7a9fb80 1 -- start start 2023-07-14T19:31:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.780+0000 7ffad7a9fb80 1 Processor -- start 2023-07-14T19:31:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb21f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb16170 0x556bdeb1f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb1f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdea06a60 con 0x556bdeb1bcc0 2023-07-14T19:31:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdea06760 con 0x556bde9716b0 2023-07-14T19:31:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bde8ea8c0 con 0x556bdeb16170 2023-07-14T19:31:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.781+0000 7ffad7a9fb80 10 client.0 _ll_register_callbacks cb 0x556bdea04700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:31:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.783+0000 7ffac9ea4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb1f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.783+0000 7ffac8ea2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb16170 0x556bdeb1f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.783+0000 7ffac9ea4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb1f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43874/0 (socket says 172.21.15.82:43874) 2023-07-14T19:31:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.783+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1832615130 learned_addr learned my addr 172.21.15.82:0/1832615130 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.783+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb21f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.796+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1832615130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb16170 msgr2=0x556bdeb1f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.796+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb16170 0x556bdeb1f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.797+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 msgr2=0x556bdeb21f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.797+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb21f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.797+0000 7ffac9ea4700 1 -- 172.21.15.82:0/1832615130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac411bfe0 con 0x556bdeb1bcc0 2023-07-14T19:31:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.803+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb1f690 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d010 tx=0x7ffabc029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5714049c6a5ef3b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.811+0000 7ffaacff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:31:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.811+0000 7ffaacff9700 1 -- 172.21.15.82:0/1832615130 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc007760 con 0x556bdeb1bcc0 2023-07-14T19:31:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.811+0000 7ffaacff9700 1 -- 172.21.15.82:0/1832615130 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7ffabc00baa0 con 0x556bdeb1bcc0 2023-07-14T19:31:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.811+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bdeae4d40 con 0x556bdeb1bcc0 2023-07-14T19:31:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.815+0000 7ffaacff9700 1 -- 172.21.15.82:0/1832615130 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffabc01dbd0 con 0x556bdeb1bcc0 2023-07-14T19:31:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.815+0000 7ffad7a9fb80 10 client.15918 fetch_fsmap learned FSMap version 35 2023-07-14T19:31:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.815+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556bde87dc00 con 0x556bdeb1bcc0 2023-07-14T19:31:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.817+0000 7ffaacff9700 1 -- 172.21.15.82:0/1832615130 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bdeae4d40 con 0x556bdeb1bcc0 2023-07-14T19:31:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.817+0000 7ffaacff9700 1 -- 172.21.15.82:0/1832615130 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ffabc01bd20 con 0x556bdeb1bcc0 2023-07-14T19:31:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.819+0000 7ffad7a9fb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 35 2023-07-14T19:31:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.819+0000 7ffad7a9fb80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:31:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.819+0000 7ffad7a9fb80 20 client.15918 populate_metadata read hostname 'smithi082' 2023-07-14T19:31:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.819+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb2f080 0x556bdeb31490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.819+0000 7ffad7a9fb80 4 client.15918 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:31:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.819+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bdea5fe50 con 0x556bdeb2f080 2023-07-14T19:31:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.823+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb2f080 0x556bdeb31490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.852+0000 7ffac96a3700 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb2f080 0x556bdeb31490 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed05a1e6657d8564 server_cookie=b3ed39c46725060d in_seq=0 out_seq=0 2023-07-14T19:31:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.852+0000 7ffaacff9700 10 client.15918 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.864+0000 7ffaacff9700 1 -- 172.21.15.82:0/1832615130 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x556bdea5fe50 con 0x556bdeb2f080 2023-07-14T19:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.864+0000 7ffaacff9700 10 client.15918 handle_command_reply: tid=0 2023-07-14T19:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.864+0000 7ffad7a9fb80 1 client.15918 shutdown 2023-07-14T19:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.896+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 msgr2=0x556bdeb1f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.896+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb1f690 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00d010 tx=0x7ffabc029040 comp rx=0 tx=0).stop 2023-07-14T19:31:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 shutdown_connections 2023-07-14T19:31:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb1bcc0 0x556bdeb1f690 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb21f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb2f080 0x556bdeb31490 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/1832615130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb16170 0x556bdeb1f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 >> 172.21.15.82:0/1832615130 conn(0x556bdeb0b620 msgr2=0x556bdeb0fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffaacff9700 0 client.15918 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.903+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 shutdown_connections 2023-07-14T19:31:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.904+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/1832615130 wait complete. 2023-07-14T19:31:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:45.904+0000 7ffad7a9fb80 20 client.15918 trim_cache size 0 max 16384 2023-07-14T19:31:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.180+0000 7ffad7a9fb80 1 Processor -- start 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 -- start start 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb15c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb16170 0x556bdeb26500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb10600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdeb1d690 con 0x556bdeb16170 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdeb2bd20 con 0x556bde9716b0 2023-07-14T19:31:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.181+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdeb2b430 con 0x556bdeb1bcc0 2023-07-14T19:31:46.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:45 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.182+0000 7ffac96a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb15c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.182+0000 7ffac96a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb15c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40226/0 (socket says 172.21.15.82:40226) 2023-07-14T19:31:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.182+0000 7ffac96a3700 1 -- 172.21.15.82:0/3213337200 learned_addr learned my addr 172.21.15.82:0/3213337200 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.182+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb10600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:45 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.187+0000 7ffac8ea2700 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb16170 0x556bdeb26500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.194+0000 7ffac9ea4700 1 -- 172.21.15.82:0/3213337200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 msgr2=0x556bdeb15c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.194+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb15c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.194+0000 7ffac9ea4700 1 -- 172.21.15.82:0/3213337200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb16170 msgr2=0x556bdeb26500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.194+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb16170 0x556bdeb26500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.194+0000 7ffac9ea4700 1 -- 172.21.15.82:0/3213337200 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bde8bb1f0 con 0x556bdeb1bcc0 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.194+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb10600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffabc02c040 tx=0x7ffabc000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22de5dc5de6b93 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.213+0000 7ffaaffff700 1 -- 172.21.15.82:0/3213337200 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc007690 con 0x556bdeb1bcc0 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.213+0000 7ffaaffff700 1 -- 172.21.15.82:0/3213337200 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffabc00baa0 con 0x556bdeb1bcc0 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.213+0000 7ffaaffff700 1 -- 172.21.15.82:0/3213337200 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc003c60 con 0x556bdeb1bcc0 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/3213337200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 msgr2=0x556bdeb10600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb10600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffabc02c040 tx=0x7ffabc000f40 comp rx=0 tx=0).stop 2023-07-14T19:31:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/3213337200 shutdown_connections 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb10600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bde9716b0 0x556bdeb15c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/3213337200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bdeb16170 0x556bdeb26500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/3213337200 >> 172.21.15.82:0/3213337200 conn(0x556bdeb0b620 msgr2=0x556bdeb19280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/3213337200 shutdown_connections 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.214+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/3213337200 wait complete. 2023-07-14T19:31:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.215+0000 7ffad7a9fb80 1 -- start start 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.215+0000 7ffad7a9fb80 1 Processor -- start 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.215+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bde9716b0 0x556bdeb18950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.215+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb214c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.215+0000 7ffad7a9fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.216+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdeb22170 con 0x556bde9716b0 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.216+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdeb222e0 con 0x556bdeb16170 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.216+0000 7ffad7a9fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bdeb171f0 con 0x556bdeb1bcc0 2023-07-14T19:31:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.216+0000 7ffad7a9fb80 10 client.0 _ll_register_callbacks cb 0x7ffabc009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:31:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.222+0000 7ffac9ea4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.222+0000 7ffac9ea4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb21a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53152/0 (socket says 172.21.15.82:53152) 2023-07-14T19:31:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.222+0000 7ffac9ea4700 1 -- 172.21.15.82:0/430780069 learned_addr learned my addr 172.21.15.82:0/430780069 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:31:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.225+0000 7ffac8ea2700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.225+0000 7ffac96a3700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bde9716b0 0x556bdeb18950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.232+0000 7ffac9ea4700 1 -- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 msgr2=0x556bdeb214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.232+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.232+0000 7ffac9ea4700 1 -- 172.21.15.82:0/430780069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bde9716b0 msgr2=0x556bdeb18950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:31:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.232+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bde9716b0 0x556bdeb18950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.232+0000 7ffac9ea4700 1 -- 172.21.15.82:0/430780069 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bde87dc00 con 0x556bdeb1bcc0 2023-07-14T19:31:46.240 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:45 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.245+0000 7ffac9ea4700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb21a60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00bc30 tx=0x7ffabc006160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd617fd1429a4892 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:31:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.298+0000 7ffaacff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:31:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.298+0000 7ffaacff9700 1 -- 172.21.15.82:0/430780069 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc031e50 con 0x556bdeb1bcc0 2023-07-14T19:31:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.298+0000 7ffaacff9700 1 -- 172.21.15.82:0/430780069 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7ffabc01d8f0 con 0x556bdeb1bcc0 2023-07-14T19:31:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.298+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bde9c0490 con 0x556bdeb1bcc0 2023-07-14T19:31:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.304+0000 7ffaacff9700 1 -- 172.21.15.82:0/430780069 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffabc003a00 con 0x556bdeb1bcc0 2023-07-14T19:31:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.305+0000 7ffad7a9fb80 10 client.25453 fetch_fsmap learned FSMap version 35 2023-07-14T19:31:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.305+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556bde893250 con 0x556bdeb1bcc0 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.305+0000 7ffaacff9700 1 -- 172.21.15.82:0/430780069 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bde9c0490 con 0x556bdeb1bcc0 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.305+0000 7ffaacff9700 1 -- 172.21.15.82:0/430780069 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ffabc0053b0 con 0x556bdeb1bcc0 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.306+0000 7ffad7a9fb80 10 client.25453 fetch_fsmap finished waiting for FSMap version 35 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.306+0000 7ffad7a9fb80 10 client.25453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.306+0000 7ffad7a9fb80 20 client.25453 populate_metadata read hostname 'smithi082' 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.306+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb82fd0 0x556bdeb853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:31:46.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.306+0000 7ffad7a9fb80 4 client.25453 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:31:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.306+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556bdea21ff0 con 0x556bdeb82fd0 2023-07-14T19:31:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.307+0000 7ffac96a3700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb82fd0 0x556bdeb853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:31:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.307+0000 7ffac96a3700 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb82fd0 0x556bdeb853e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad39cdc027017860 server_cookie=6d304c6947118998 in_seq=0 out_seq=0 2023-07-14T19:31:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:46.307+0000 7ffaacff9700 10 client.25453 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:47.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:46 smithi133 ceph-mon[128894]: pgmap v100: 97 pgs: 97 active+clean; 744 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 581 op/s 2023-07-14T19:31:47.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:46 smithi161 ceph-mon[129184]: pgmap v100: 97 pgs: 97 active+clean; 744 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 581 op/s 2023-07-14T19:31:47.293 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:46 smithi082 ceph-mon[127461]: pgmap v100: 97 pgs: 97 active+clean; 744 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 581 op/s 2023-07-14T19:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.480+0000 7ffaacff9700 1 -- 172.21.15.82:0/430780069 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556bdea21ff0 con 0x556bdeb82fd0 2023-07-14T19:31:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.480+0000 7ffaacff9700 10 client.25453 handle_command_reply: tid=0 2023-07-14T19:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.480+0000 7ffad7a9fb80 1 client.25453 shutdown 2023-07-14T19:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 msgr2=0x556bdeb21a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb21a60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00bc30 tx=0x7ffabc006160 comp rx=0 tx=0).stop 2023-07-14T19:31:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 shutdown_connections 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556bdeb82fd0 0x556bdeb853e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556bdeb1bcc0 0x556bdeb21a60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556bde9716b0 0x556bdeb18950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 --2- 172.21.15.82:0/430780069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556bdeb16170 0x556bdeb214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 >> 172.21.15.82:0/430780069 conn(0x556bdeb0b620 msgr2=0x556bdeb1c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.481+0000 7ffaacff9700 0 client.25453 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:31:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.482+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 shutdown_connections 2023-07-14T19:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.482+0000 7ffad7a9fb80 1 -- 172.21.15.82:0/430780069 wait complete. 2023-07-14T19:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.483+0000 7ffad7a9fb80 20 client.25453 trim_cache size 0 max 16384 2023-07-14T19:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:31:47.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:31:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:31:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.611+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ffab00a14d0 msgr2=0x7ffab00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.611+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ffab00a14d0 0x7ffab00a3980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffabc021000 tx=0x7ffabc01d780 comp rx=0 tx=0).stop 2023-07-14T19:31:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.611+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 msgr2=0x7ffac41aa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:31:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.611+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 0x7ffac41aa960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffac000d980 tx=0x7ffac000dc90 comp rx=0 tx=0).stop 2023-07-14T19:31:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 shutdown_connections 2023-07-14T19:31:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ffab00a14d0 0x7ffab00a3980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffac410e1a0 0x7ffac41a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffac406dc90 0x7ffac41aa960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 --2- 172.21.15.82:0/1467957558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac406e640 0x7ffac41aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:31:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 >> 172.21.15.82:0/1467957558 conn(0x7ffac4071000 msgr2=0x7ffac406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:31:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 shutdown_connections 2023-07-14T19:31:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:31:47.612+0000 7ffaca6a5700 1 -- 172.21.15.82:0/1467957558 wait complete. 2023-07-14T19:31:49.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:48 smithi133 ceph-mon[128894]: pgmap v101: 97 pgs: 97 active+clean; 747 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2023-07-14T19:31:49.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:48 smithi082 ceph-mon[127461]: pgmap v101: 97 pgs: 97 active+clean; 747 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2023-07-14T19:31:49.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:48 smithi161 ceph-mon[129184]: pgmap v101: 97 pgs: 97 active+clean; 747 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2023-07-14T19:31:50.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:50 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:50.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:50 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:50.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:50 smithi133 ceph-mon[128894]: pgmap v102: 97 pgs: 97 active+clean; 752 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2023-07-14T19:31:50.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:50 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:50.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:50 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:50.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:50 smithi082 ceph-mon[127461]: pgmap v102: 97 pgs: 97 active+clean; 752 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2023-07-14T19:31:50.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:50 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:50.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:50 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:50.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:50 smithi161 ceph-mon[129184]: pgmap v102: 97 pgs: 97 active+clean; 752 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2023-07-14T19:31:51.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:51 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:51.719 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:51 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:51.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:51 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:31:52.181 INFO:tasks.workunit.client.0.smithi082.stdout: 213 2422363 497 1716377 380 723945 1054 2023-07-14T19:31:52.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:52 smithi133 ceph-mon[128894]: pgmap v103: 97 pgs: 97 active+clean; 746 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 438 op/s 2023-07-14T19:31:52.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:52 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:31:52.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:52 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:31:52.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:52 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:52.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:52 smithi082 ceph-mon[127461]: pgmap v103: 97 pgs: 97 active+clean; 746 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 438 op/s 2023-07-14T19:31:52.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:52 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:31:52.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:52 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:31:52.714 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:52 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:52.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:52 smithi161 ceph-mon[129184]: pgmap v103: 97 pgs: 97 active+clean; 746 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 438 op/s 2023-07-14T19:31:52.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:52 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:31:52.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:52 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:31:52.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:52 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:31:54.933 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:54 smithi133 ceph-mon[128894]: pgmap v104: 97 pgs: 97 active+clean; 749 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 464 op/s 2023-07-14T19:31:54.964 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:54 smithi082 ceph-mon[127461]: pgmap v104: 97 pgs: 97 active+clean; 749 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 464 op/s 2023-07-14T19:31:54.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:54 smithi161 ceph-mon[129184]: pgmap v104: 97 pgs: 97 active+clean; 749 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 464 op/s 2023-07-14T19:31:56.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:56 smithi161 ceph-mon[129184]: pgmap v105: 97 pgs: 97 active+clean; 830 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 859 op/s 2023-07-14T19:31:57.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:56 smithi133 ceph-mon[128894]: pgmap v105: 97 pgs: 97 active+clean; 830 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 859 op/s 2023-07-14T19:31:57.229 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:56 smithi082 ceph-mon[127461]: pgmap v105: 97 pgs: 97 active+clean; 830 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.2 MiB/s wr, 859 op/s 2023-07-14T19:31:59.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:31:58 smithi133 ceph-mon[128894]: pgmap v106: 97 pgs: 97 active+clean; 863 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-07-14T19:31:59.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:31:58 smithi082 ceph-mon[127461]: pgmap v106: 97 pgs: 97 active+clean; 863 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-07-14T19:31:59.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:31:58 smithi161 ceph-mon[129184]: pgmap v106: 97 pgs: 97 active+clean; 863 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-07-14T19:32:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 -- 172.21.15.82:0/4275325398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 msgr2=0x7f56f410b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 --2- 172.21.15.82:0/4275325398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f410b220 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00b0a0 tx=0x7f56ec00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 -- 172.21.15.82:0/4275325398 shutdown_connections 2023-07-14T19:32:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 --2- 172.21.15.82:0/4275325398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f410b220 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 --2- 172.21.15.82:0/4275325398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f41044a0 0x7f56f4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 --2- 172.21.15.82:0/4275325398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f40fee50 0x7f56f40ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 -- 172.21.15.82:0/4275325398 >> 172.21.15.82:0/4275325398 conn(0x7f56f40753a0 msgr2=0x7f56f40757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 -- 172.21.15.82:0/4275325398 shutdown_connections 2023-07-14T19:32:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.911+0000 7f56f94a6700 1 -- 172.21.15.82:0/4275325398 wait complete. 2023-07-14T19:32:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 Processor -- start 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 -- start start 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f40fee50 0x7f56f4196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f41044a0 0x7f56f41a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f41a2370 con 0x7f56f40ff800 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f41a24e0 con 0x7f56f40fee50 2023-07-14T19:32:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f94a6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f41a26a0 con 0x7f56f41044a0 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f4197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43900/0 (socket says 172.21.15.82:43900) 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f27fc700 1 -- 172.21.15.82:0/3816367927 learned_addr learned my addr 172.21.15.82:0/3816367927 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.912+0000 7f56f37fe700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f41044a0 0x7f56f41a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f40fee50 0x7f56f4196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f27fc700 1 -- 172.21.15.82:0/3816367927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f41044a0 msgr2=0x7f56f41a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f41044a0 0x7f56f41a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f27fc700 1 -- 172.21.15.82:0/3816367927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f40fee50 msgr2=0x7f56f4196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f40fee50 0x7f56f4196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f27fc700 1 -- 172.21.15.82:0/3816367927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec009d00 con 0x7f56f40ff800 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f4197410 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f56e800e9a0 tx=0x7f56e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56dbfff700 1 -- 172.21.15.82:0/3816367927 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8009790 con 0x7f56f40ff800 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56dbfff700 1 -- 172.21.15.82:0/3816367927 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e8010430 con 0x7f56f40ff800 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f41a2980 con 0x7f56f40ff800 2023-07-14T19:32:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56dbfff700 1 -- 172.21.15.82:0/3816367927 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8018680 con 0x7f56f40ff800 2023-07-14T19:32:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.913+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f41a2e90 con 0x7f56f40ff800 2023-07-14T19:32:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.915+0000 7f56dbfff700 1 -- 172.21.15.82:0/3816367927 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f56e80105a0 con 0x7f56f40ff800 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.916+0000 7f56dbfff700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f56dc0a12c0 0x7f56dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.916+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f56dc0a12c0 0x7f56dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.916+0000 7f56dbfff700 1 -- 172.21.15.82:0/3816367927 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f56e8014070 con 0x7f56f40ff800 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.916+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f56dc0a12c0 0x7f56dc0a3770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005950 tx=0x7f56e400b410 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.916+0000 7f570563eb80 1 Processor -- start 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.916+0000 7f570563eb80 1 -- start start 2023-07-14T19:32:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f570563eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3465f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f570563eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f570563eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34659f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f570563eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3455ec50 con 0x55fb344b56b0 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f570563eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb345b19e0 con 0x55fb3465fa90 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f570563eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb345aae70 con 0x55fb3465ff00 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3465f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3465f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43906/0 (socket says 172.21.15.82:43906) 2023-07-14T19:32:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 -- 172.21.15.82:0/2992768201 learned_addr learned my addr 172.21.15.82:0/2992768201 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f27fc700 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f37fe700 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34659f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 -- 172.21.15.82:0/2992768201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 msgr2=0x55fb34659f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34659f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 -- 172.21.15.82:0/2992768201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 msgr2=0x55fb346599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 -- 172.21.15.82:0/2992768201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb343d8d80 con 0x55fb344b56b0 2023-07-14T19:32:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f37fe700 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34659f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3465f550 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f56e4010e60 tx=0x7f56e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f44f1eb91ba766b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56d97fa700 1 -- 172.21.15.82:0/2992768201 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e401e620 con 0x55fb344b56b0 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56d97fa700 1 -- 172.21.15.82:0/2992768201 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e4015070 con 0x55fb344b56b0 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.917+0000 7f56d97fa700 1 -- 172.21.15.82:0/2992768201 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e401c980 con 0x55fb344b56b0 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 -- 172.21.15.82:0/2992768201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 msgr2=0x55fb3465f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3465f550 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f56e4010e60 tx=0x7f56e4013000 comp rx=0 tx=0).stop 2023-07-14T19:32:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 -- 172.21.15.82:0/2992768201 shutdown_connections 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34659f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3465f550 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 --2- 172.21.15.82:0/2992768201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 -- 172.21.15.82:0/2992768201 >> 172.21.15.82:0/2992768201 conn(0x55fb3464f3d0 msgr2=0x55fb34654530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 -- 172.21.15.82:0/2992768201 shutdown_connections 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 -- 172.21.15.82:0/2992768201 wait complete. 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.918+0000 7f570563eb80 1 -- start start 2023-07-14T19:32:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 Processor -- start 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb34665df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb34663070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34663710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3454aa60 con 0x55fb3465fa90 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3454a760 con 0x55fb344b56b0 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3442e8c0 con 0x55fb3465ff00 2023-07-14T19:32:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 10 client.0 _ll_register_callbacks cb 0x55fb34548700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb34663070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb34663070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43914/0 (socket says 172.21.15.82:43914) 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 -- 172.21.15.82:0/3660152017 learned_addr learned my addr 172.21.15.82:0/3660152017 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f37fe700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34663710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb34665df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 -- 172.21.15.82:0/3660152017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 msgr2=0x55fb34663710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34663710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 -- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 msgr2=0x55fb34665df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb34665df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 -- 172.21.15.82:0/3660152017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4010b10 con 0x55fb3465fa90 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb34663070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f56e8096a10 tx=0x7f56e809e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7befdc93915f05b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56d6ff5700 1 -- 172.21.15.82:0/3660152017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8099690 con 0x55fb3465fa90 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f56d6ff5700 1 -- 172.21.15.82:0/3660152017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f56e80a1f60 con 0x55fb3465fa90 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.919+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb34504490 con 0x55fb3465fa90 2023-07-14T19:32:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f56d6ff5700 1 -- 172.21.15.82:0/3660152017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e80a2120 con 0x55fb3465fa90 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 10 client.15936 fetch_fsmap learned FSMap version 36 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb343c1c00 con 0x55fb3465fa90 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f56d6ff5700 1 -- 172.21.15.82:0/3660152017 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e80c8080 con 0x55fb3465fa90 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f56d6ff5700 1 -- 172.21.15.82:0/3660152017 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f56e8098f00 con 0x55fb3465fa90 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:32:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 20 client.15936 populate_metadata read hostname 'smithi082' 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 4 client.15936 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb34628d40 con 0x55fb346732b0 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.920+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f56d6ff5700 10 client.15936 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346756c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3435cd579fa1daa5 server_cookie=84a268eb72a52a2b in_seq=0 out_seq=0 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f56d6ff5700 1 -- 172.21.15.82:0/3660152017 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55fb34628d40 con 0x55fb346732b0 2023-07-14T19:32:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f56d6ff5700 10 client.15936 handle_command_reply: tid=0 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 client.15936 shutdown 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 msgr2=0x55fb34663070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb34663070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f56e8096a10 tx=0x7f56e809e080 comp rx=0 tx=0).stop 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 shutdown_connections 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346756c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34663710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f56d6ff5700 0 client.15936 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb34665df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 --2- 172.21.15.82:0/3660152017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb34663070 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 >> 172.21.15.82:0/3660152017 conn(0x55fb3464f3d0 msgr2=0x55fb34653c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 shutdown_connections 2023-07-14T19:32:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 1 -- 172.21.15.82:0/3660152017 wait complete. 2023-07-14T19:32:00.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.921+0000 7f570563eb80 20 client.15936 trim_cache size 0 max 16384 2023-07-14T19:32:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.940+0000 7f570563eb80 1 Processor -- start 2023-07-14T19:32:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.940+0000 7f570563eb80 1 -- start start 2023-07-14T19:32:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f570563eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3466a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f570563eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f570563eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34665990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f570563eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3465a7c0 con 0x55fb344b56b0 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f570563eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34670960 con 0x55fb3465fa90 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f570563eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb34661620 con 0x55fb3465ff00 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3466a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3466a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43920/0 (socket says 172.21.15.82:43920) 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f37fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34665990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 -- 172.21.15.82:0/3290014777 learned_addr learned my addr 172.21.15.82:0/3290014777 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:00.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f27fc700 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 -- 172.21.15.82:0/3290014777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 msgr2=0x55fb34665990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34665990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 -- 172.21.15.82:0/3290014777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 msgr2=0x55fb346543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 -- 172.21.15.82:0/3290014777 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb343ff1f0 con 0x55fb344b56b0 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f37fe700 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34665990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3466a400 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f56e4010e60 tx=0x7f56e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4675acd6959dc15 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.941+0000 7f56d9ffb700 1 -- 172.21.15.82:0/3290014777 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e401c3b0 con 0x55fb344b56b0 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f56d9ffb700 1 -- 172.21.15.82:0/3290014777 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e402e040 con 0x55fb344b56b0 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f56d9ffb700 1 -- 172.21.15.82:0/3290014777 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4004690 con 0x55fb344b56b0 2023-07-14T19:32:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 -- 172.21.15.82:0/3290014777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 msgr2=0x55fb3466a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3466a400 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f56e4010e60 tx=0x7f56e4016000 comp rx=0 tx=0).stop 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 -- 172.21.15.82:0/3290014777 shutdown_connections 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb34665990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb344b56b0 0x55fb3466a400 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 --2- 172.21.15.82:0/3290014777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb3465fa90 0x55fb346543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 -- 172.21.15.82:0/3290014777 >> 172.21.15.82:0/3290014777 conn(0x55fb3464f3d0 msgr2=0x55fb3465d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 -- 172.21.15.82:0/3290014777 shutdown_connections 2023-07-14T19:32:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.942+0000 7f570563eb80 1 -- 172.21.15.82:0/3290014777 wait complete. 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 -- start start 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 Processor -- start 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb3465c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb3465af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb3465b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3465b9f0 con 0x55fb3465fa90 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3465bb60 con 0x55fb344b56b0 2023-07-14T19:32:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb3465bd20 con 0x55fb3465ff00 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f570563eb80 10 client.0 _ll_register_callbacks cb 0x55fb34548700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f56f27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb3465af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f56f2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb3465c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f56f2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb3465c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40274/0 (socket says 172.21.15.82:40274) 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f56f2ffd700 1 -- 172.21.15.82:0/2769925019 learned_addr learned my addr 172.21.15.82:0/2769925019 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.943+0000 7f56f37fe700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb3465b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f37fe700 1 -- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 msgr2=0x55fb3465c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f37fe700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb3465c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f37fe700 1 -- 172.21.15.82:0/2769925019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 msgr2=0x55fb3465af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f37fe700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb3465af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f37fe700 1 -- 172.21.15.82:0/2769925019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f4068430 con 0x55fb3465ff00 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f27fc700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb3465af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56f37fe700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb3465b4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0062a0 tx=0x7f56ec004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380c93892322bf0a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:32:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56d6ff5700 1 -- 172.21.15.82:0/2769925019 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00e040 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e4015070 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56d6ff5700 1 -- 172.21.15.82:0/2769925019 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f56ec0076e0 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56d6ff5700 1 -- 172.21.15.82:0/2769925019 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56ec0078a0 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f570563eb80 10 client.25468 fetch_fsmap learned FSMap version 36 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb343d8d80 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.944+0000 7f56d6ff5700 1 -- 172.21.15.82:0/2769925019 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e4015070 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f56d6ff5700 1 -- 172.21.15.82:0/2769925019 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f56ec00b0d0 con 0x55fb3465ff00 2023-07-14T19:32:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f570563eb80 10 client.25468 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:32:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f570563eb80 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:32:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f570563eb80 20 client.25468 populate_metadata read hostname 'smithi082' 2023-07-14T19:32:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f570563eb80 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f570563eb80 4 client.25468 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:32:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb345a3e50 con 0x55fb346732b0 2023-07-14T19:32:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f56d6ff5700 10 client.25468 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:00.945+0000 7f56f2ffd700 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346ca470 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94fa56faaae1b844 server_cookie=933ca89399ce97b5 in_seq=0 out_seq=0 2023-07-14T19:32:01.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:00 smithi133 ceph-mon[128894]: pgmap v107: 97 pgs: 97 active+clean; 864 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-07-14T19:32:01.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:00 smithi082 ceph-mon[127461]: pgmap v107: 97 pgs: 97 active+clean; 864 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-07-14T19:32:01.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:00 smithi161 ceph-mon[129184]: pgmap v107: 97 pgs: 97 active+clean; 864 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-07-14T19:32:02.181 INFO:tasks.workunit.client.0.smithi082.stdout: 218 1687066 255 1189153 377 520448 735 2023-07-14T19:32:03.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:02 smithi133 ceph-mon[128894]: pgmap v108: 97 pgs: 97 active+clean; 870 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 941 op/s 2023-07-14T19:32:03.231 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:02 smithi082 ceph-mon[127461]: pgmap v108: 97 pgs: 97 active+clean; 870 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 941 op/s 2023-07-14T19:32:03.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:02 smithi161 ceph-mon[129184]: pgmap v108: 97 pgs: 97 active+clean; 870 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 941 op/s 2023-07-14T19:32:05.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:04 smithi133 ceph-mon[128894]: pgmap v109: 97 pgs: 97 active+clean; 870 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 747 op/s 2023-07-14T19:32:05.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:04 smithi082 ceph-mon[127461]: pgmap v109: 97 pgs: 97 active+clean; 870 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 747 op/s 2023-07-14T19:32:05.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:04 smithi161 ceph-mon[129184]: pgmap v109: 97 pgs: 97 active+clean; 870 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 747 op/s 2023-07-14T19:32:07.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:06 smithi133 ceph-mon[128894]: pgmap v110: 97 pgs: 97 active+clean; 875 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 11 MiB/s wr, 969 op/s 2023-07-14T19:32:07.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:06 smithi082 ceph-mon[127461]: pgmap v110: 97 pgs: 97 active+clean; 875 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 11 MiB/s wr, 969 op/s 2023-07-14T19:32:07.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:06 smithi161 ceph-mon[129184]: pgmap v110: 97 pgs: 97 active+clean; 875 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 11 MiB/s wr, 969 op/s 2023-07-14T19:32:09.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:08 smithi133 ceph-mon[128894]: pgmap v111: 97 pgs: 97 active+clean; 875 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 608 op/s 2023-07-14T19:32:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:08 smithi082 ceph-mon[127461]: pgmap v111: 97 pgs: 97 active+clean; 875 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 608 op/s 2023-07-14T19:32:09.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:08 smithi161 ceph-mon[129184]: pgmap v111: 97 pgs: 97 active+clean; 875 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 608 op/s 2023-07-14T19:32:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:10 smithi133 ceph-mon[128894]: pgmap v112: 97 pgs: 97 active+clean; 875 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 428 op/s 2023-07-14T19:32:11.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:10 smithi082 ceph-mon[127461]: pgmap v112: 97 pgs: 97 active+clean; 875 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 428 op/s 2023-07-14T19:32:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:10 smithi161 ceph-mon[129184]: pgmap v112: 97 pgs: 97 active+clean; 875 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 428 op/s 2023-07-14T19:32:12.181 INFO:tasks.workunit.client.0.smithi082.stdout: 229 839537 700 593921 693 270353 1008 2023-07-14T19:32:13.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:12 smithi133 ceph-mon[128894]: pgmap v113: 97 pgs: 97 active+clean; 890 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 473 op/s 2023-07-14T19:32:13.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:12 smithi082 ceph-mon[127461]: pgmap v113: 97 pgs: 97 active+clean; 890 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 473 op/s 2023-07-14T19:32:13.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:12 smithi161 ceph-mon[129184]: pgmap v113: 97 pgs: 97 active+clean; 890 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 473 op/s 2023-07-14T19:32:15.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:14 smithi133 ceph-mon[128894]: pgmap v114: 97 pgs: 97 active+clean; 890 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 437 op/s 2023-07-14T19:32:15.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:14 smithi082 ceph-mon[127461]: pgmap v114: 97 pgs: 97 active+clean; 890 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 437 op/s 2023-07-14T19:32:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:14 smithi161 ceph-mon[129184]: pgmap v114: 97 pgs: 97 active+clean; 890 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 437 op/s 2023-07-14T19:32:17.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:16 smithi133 ceph-mon[128894]: pgmap v115: 97 pgs: 97 active+clean; 901 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 471 op/s 2023-07-14T19:32:17.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:16 smithi082 ceph-mon[127461]: pgmap v115: 97 pgs: 97 active+clean; 901 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 471 op/s 2023-07-14T19:32:17.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:16 smithi161 ceph-mon[129184]: pgmap v115: 97 pgs: 97 active+clean; 901 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 471 op/s 2023-07-14T19:32:19.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:18 smithi133 ceph-mon[128894]: pgmap v116: 97 pgs: 97 active+clean; 901 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2023-07-14T19:32:19.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:18 smithi082 ceph-mon[127461]: pgmap v116: 97 pgs: 97 active+clean; 901 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2023-07-14T19:32:19.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:18 smithi161 ceph-mon[129184]: pgmap v116: 97 pgs: 97 active+clean; 901 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 193 op/s 2023-07-14T19:32:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.767+0000 7f56d6ff5700 1 -- 172.21.15.82:0/2769925019 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb345a3e50 con 0x55fb346732b0 2023-07-14T19:32:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.767+0000 7f56d6ff5700 10 client.25468 handle_command_reply: tid=0 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.767+0000 7f570563eb80 1 client.25468 shutdown 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 msgr2=0x55fb3465b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb3465b4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0062a0 tx=0x7f56ec004c40 comp rx=0 tx=0).stop 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 shutdown_connections 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fb346732b0 0x55fb346ca470 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fb3465ff00 0x55fb3465b4b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb344b56b0 0x55fb3465c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 --2- 172.21.15.82:0/2769925019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fb3465fa90 0x55fb3465af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 >> 172.21.15.82:0/2769925019 conn(0x55fb3464f3d0 msgr2=0x55fb34660300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f56d6ff5700 0 client.25468 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 shutdown_connections 2023-07-14T19:32:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.768+0000 7f570563eb80 1 -- 172.21.15.82:0/2769925019 wait complete. 2023-07-14T19:32:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:32:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:32:19.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:32:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:32:19.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.776+0000 7f570563eb80 20 client.25468 trim_cache size 0 max 16384 2023-07-14T19:32:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.810+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f56dc0a12c0 msgr2=0x7f56dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.810+0000 7f56f94a6700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f56dc0a12c0 0x7f56dc0a3770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005950 tx=0x7f56e400b410 comp rx=0 tx=0).stop 2023-07-14T19:32:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.810+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 msgr2=0x7f56f4197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.810+0000 7f56f94a6700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f4197410 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f56e800e9a0 tx=0x7f56e800ecb0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.815+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 shutdown_connections 2023-07-14T19:32:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.815+0000 7f56f94a6700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f56dc0a12c0 0x7f56dc0a3770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.815+0000 7f56f94a6700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56f41044a0 0x7f56f41a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.815+0000 7f56f94a6700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f40fee50 0x7f56f4196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.815+0000 7f56f94a6700 1 --2- 172.21.15.82:0/3816367927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56f40ff800 0x7f56f4197410 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.815+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 >> 172.21.15.82:0/3816367927 conn(0x7f56f40753a0 msgr2=0x7f56f40fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.820+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 shutdown_connections 2023-07-14T19:32:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:19.831+0000 7f56f94a6700 1 -- 172.21.15.82:0/3816367927 wait complete. 2023-07-14T19:32:21.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:20 smithi133 ceph-mon[128894]: pgmap v117: 97 pgs: 97 active+clean; 935 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 307 op/s 2023-07-14T19:32:21.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:20 smithi082 ceph-mon[127461]: pgmap v117: 97 pgs: 97 active+clean; 935 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 307 op/s 2023-07-14T19:32:21.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:20 smithi161 ceph-mon[129184]: pgmap v117: 97 pgs: 97 active+clean; 935 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 307 op/s 2023-07-14T19:32:22.196 INFO:tasks.workunit.client.0.smithi082.stdout: 244 1479547 620 1043112 652 480910 714 2023-07-14T19:32:23.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:22 smithi133 ceph-mon[128894]: pgmap v118: 97 pgs: 97 active+clean; 1007 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 567 op/s 2023-07-14T19:32:23.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:22 smithi082 ceph-mon[127461]: pgmap v118: 97 pgs: 97 active+clean; 1007 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 567 op/s 2023-07-14T19:32:23.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:22 smithi161 ceph-mon[129184]: pgmap v118: 97 pgs: 97 active+clean; 1007 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 567 op/s 2023-07-14T19:32:25.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:24 smithi082 ceph-mon[127461]: pgmap v119: 97 pgs: 97 active+clean; 1018 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 566 op/s 2023-07-14T19:32:25.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:24 smithi161 ceph-mon[129184]: pgmap v119: 97 pgs: 97 active+clean; 1018 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 566 op/s 2023-07-14T19:32:25.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:24 smithi133 ceph-mon[128894]: pgmap v119: 97 pgs: 97 active+clean; 1018 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 566 op/s 2023-07-14T19:32:27.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:27 smithi133 ceph-mon[128894]: pgmap v120: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 631 op/s 2023-07-14T19:32:27.465 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:27 smithi082 ceph-mon[127461]: pgmap v120: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 631 op/s 2023-07-14T19:32:27.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:27 smithi161 ceph-mon[129184]: pgmap v120: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 631 op/s 2023-07-14T19:32:29.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:29 smithi133 ceph-mon[128894]: pgmap v121: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 590 op/s 2023-07-14T19:32:29.472 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:29 smithi082 ceph-mon[127461]: pgmap v121: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 590 op/s 2023-07-14T19:32:29.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:29 smithi161 ceph-mon[129184]: pgmap v121: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 590 op/s 2023-07-14T19:32:31.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:31 smithi133 ceph-mon[128894]: pgmap v122: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 596 op/s 2023-07-14T19:32:31.463 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:31 smithi082 ceph-mon[127461]: pgmap v122: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 596 op/s 2023-07-14T19:32:31.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:31 smithi161 ceph-mon[129184]: pgmap v122: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 596 op/s 2023-07-14T19:32:32.191 INFO:tasks.workunit.client.0.smithi082.stdout: 260 2442872 808 1719309 997 820871 1245 2023-07-14T19:32:33.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:33 smithi133 ceph-mon[128894]: pgmap v123: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.8 MiB/s wr, 484 op/s 2023-07-14T19:32:33.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:33 smithi082 ceph-mon[127461]: pgmap v123: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.8 MiB/s wr, 484 op/s 2023-07-14T19:32:33.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:33 smithi161 ceph-mon[129184]: pgmap v123: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.8 MiB/s wr, 484 op/s 2023-07-14T19:32:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.277+0000 7f24acd7c700 1 -- 172.21.15.82:0/112759040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a810e3d0 msgr2=0x7f24a810e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.277+0000 7f24acd7c700 1 --2- 172.21.15.82:0/112759040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a810e3d0 0x7f24a810e7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f24980098f0 tx=0x7f2498009c00 comp rx=0 tx=0).stop 2023-07-14T19:32:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.278+0000 7f24acd7c700 1 -- 172.21.15.82:0/112759040 shutdown_connections 2023-07-14T19:32:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.278+0000 7f24acd7c700 1 --2- 172.21.15.82:0/112759040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a806d4d0 0x7f24a811b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.278+0000 7f24acd7c700 1 --2- 172.21.15.82:0/112759040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a810e3d0 0x7f24a810e7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.278+0000 7f24acd7c700 1 --2- 172.21.15.82:0/112759040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806ca40 0x7f24a806ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.278+0000 7f24acd7c700 1 -- 172.21.15.82:0/112759040 >> 172.21.15.82:0/112759040 conn(0x7f24a806bea0 msgr2=0x7f24a8071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.284+0000 7f24acd7c700 1 -- 172.21.15.82:0/112759040 shutdown_connections 2023-07-14T19:32:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.284+0000 7f24acd7c700 1 -- 172.21.15.82:0/112759040 wait complete. 2023-07-14T19:32:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.285+0000 7f24acd7c700 1 Processor -- start 2023-07-14T19:32:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.285+0000 7f24acd7c700 1 -- start start 2023-07-14T19:32:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.286+0000 7f24acd7c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a806ca40 0x7f24a81aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.286+0000 7f24acd7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 0x7f24a81ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.286+0000 7f24acd7c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a810e3d0 0x7f24a81a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.286+0000 7f24acd7c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81a51f0 con 0x7f24a806d4d0 2023-07-14T19:32:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.286+0000 7f24acd7c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81a5360 con 0x7f24a810e3d0 2023-07-14T19:32:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.286+0000 7f24acd7c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81a5520 con 0x7f24a806ca40 2023-07-14T19:32:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.291+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a810e3d0 0x7f24a81a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.291+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 0x7f24a81ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.291+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 0x7f24a81ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43942/0 (socket says 172.21.15.82:43942) 2023-07-14T19:32:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.291+0000 7f24a5d9b700 1 -- 172.21.15.82:0/511942913 learned_addr learned my addr 172.21.15.82:0/511942913 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.292+0000 7f24a5d9b700 1 -- 172.21.15.82:0/511942913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a806ca40 msgr2=0x7f24a81aac80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T19:32:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.292+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a806ca40 0x7f24a81aac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.292+0000 7f24a5d9b700 1 -- 172.21.15.82:0/511942913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a810e3d0 msgr2=0x7f24a81a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.292+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a810e3d0 0x7f24a81a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.292+0000 7f24a5d9b700 1 -- 172.21.15.82:0/511942913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0009d00 con 0x7f24a806d4d0 2023-07-14T19:32:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.293+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 0x7f24a81ab1c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e3c0 tx=0x7f24a000e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.293+0000 7f24977fe700 1 -- 172.21.15.82:0/511942913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0018070 con 0x7f24a806d4d0 2023-07-14T19:32:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.293+0000 7f24977fe700 1 -- 172.21.15.82:0/511942913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a000f040 con 0x7f24a806d4d0 2023-07-14T19:32:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.293+0000 7f24977fe700 1 -- 172.21.15.82:0/511942913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0004480 con 0x7f24a806d4d0 2023-07-14T19:32:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.293+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498009520 con 0x7f24a806d4d0 2023-07-14T19:32:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.294+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a810ef50 con 0x7f24a806d4d0 2023-07-14T19:32:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.299+0000 7f24b8f14b80 1 Processor -- start 2023-07-14T19:32:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.305+0000 7f24977fe700 1 -- 172.21.15.82:0/511942913 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f24a0004bd0 con 0x7f24a806d4d0 2023-07-14T19:32:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.306+0000 7f24977fe700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f24900a14d0 0x7f24900a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.306+0000 7f24977fe700 1 -- 172.21.15.82:0/511942913 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f24a00cc980 con 0x7f24a806d4d0 2023-07-14T19:32:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.306+0000 7f24b8f14b80 1 -- start start 2023-07-14T19:32:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc13696b0 0x563bc1513550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 0x563bc150d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc150df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc1412c50 con 0x563bc13696b0 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc14659e0 con 0x563bc1513a90 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc145ee70 con 0x563bc1513f00 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 0x563bc150d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 0x563bc150d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40296/0 (socket says 172.21.15.82:40296) 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24a5d9b700 1 -- 172.21.15.82:0/84759771 learned_addr learned my addr 172.21.15.82:0/84759771 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.307+0000 7f24a6d9d700 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc150df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a5d9b700 1 -- 172.21.15.82:0/84759771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 msgr2=0x563bc150df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc150df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a5d9b700 1 -- 172.21.15.82:0/84759771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc13696b0 msgr2=0x563bc1513550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:32:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc13696b0 0x563bc1513550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a5d9b700 1 -- 172.21.15.82:0/84759771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bc128cd80 con 0x563bc1513a90 2023-07-14T19:32:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 0x563bc150d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24a00a19a0 tx=0x7f24a00996d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97231a8ea6e90da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f24a6d9d700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f24900a14d0 0x7f24900a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f2494ff9700 1 -- 172.21.15.82:0/84759771 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0099b10 con 0x563bc1513a90 2023-07-14T19:32:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f2494ff9700 1 -- 172.21.15.82:0/84759771 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a009db40 con 0x563bc1513a90 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.308+0000 7f2494ff9700 1 -- 172.21.15.82:0/84759771 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0099000 con 0x563bc1513a90 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 -- 172.21.15.82:0/84759771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 msgr2=0x563bc150d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 0x563bc150d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24a00a19a0 tx=0x7f24a00996d0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 -- 172.21.15.82:0/84759771 shutdown_connections 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc150df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc13696b0 0x563bc1513550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/84759771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513a90 0x563bc150d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.310+0000 7f24b8f14b80 1 -- 172.21.15.82:0/84759771 >> 172.21.15.82:0/84759771 conn(0x563bc15033d0 msgr2=0x563bc1508530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.311+0000 7f24a6d9d700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f24900a14d0 0x7f24900a3980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x563bc15142d0 tx=0x7f249c0047c0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.328+0000 7f24b8f14b80 1 -- 172.21.15.82:0/84759771 shutdown_connections 2023-07-14T19:32:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.328+0000 7f24b8f14b80 1 -- 172.21.15.82:0/84759771 wait complete. 2023-07-14T19:32:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 -- start start 2023-07-14T19:32:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 Processor -- start 2023-07-14T19:32:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc151ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc13fea60 con 0x563bc1513a90 2023-07-14T19:32:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc13fe760 con 0x563bc1517050 2023-07-14T19:32:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.329+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc12e28c0 con 0x563bc1513f00 2023-07-14T19:32:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.330+0000 7f24b8f14b80 10 client.0 _ll_register_callbacks cb 0x563bc13fc700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:32:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.333+0000 7f24a659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc151ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.334+0000 7f24a659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc151ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43954/0 (socket says 172.21.15.82:43954) 2023-07-14T19:32:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.334+0000 7f24a659c700 1 -- 172.21.15.82:0/639103217 learned_addr learned my addr 172.21.15.82:0/639103217 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.339+0000 7f24a6d9d700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.339+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.339+0000 7f24a5d9b700 1 -- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 msgr2=0x563bc15174c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.339+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15174c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.339+0000 7f24a5d9b700 1 -- 172.21.15.82:0/639103217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 msgr2=0x563bc151ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.339+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc151ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.340+0000 7f24a5d9b700 1 -- 172.21.15.82:0/639103217 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a810ef50 con 0x563bc1513f00 2023-07-14T19:32:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.346+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24a00a19a0 tx=0x7f24a009af70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d54b890151a5d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.358+0000 7f248dffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:32:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.358+0000 7f248dffb700 1 -- 172.21.15.82:0/639103217 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00b2260 con 0x563bc1513f00 2023-07-14T19:32:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.358+0000 7f248dffb700 1 -- 172.21.15.82:0/639103217 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f24a0099b10 con 0x563bc1513f00 2023-07-14T19:32:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.358+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bc13b8490 con 0x563bc1513f00 2023-07-14T19:32:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.368+0000 7f248dffb700 1 -- 172.21.15.82:0/639103217 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a0099f80 con 0x563bc1513f00 2023-07-14T19:32:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.389+0000 7f24b8f14b80 10 client.25477 fetch_fsmap learned FSMap version 36 2023-07-14T19:32:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.389+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563bc1275c00 con 0x563bc1513f00 2023-07-14T19:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.392+0000 7f248dffb700 1 -- 172.21.15.82:0/639103217 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bc13b8490 con 0x563bc1513f00 2023-07-14T19:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.392+0000 7f248dffb700 1 -- 172.21.15.82:0/639103217 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f24a00c5020 con 0x563bc1513f00 2023-07-14T19:32:34.394 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.393+0000 7f24b8f14b80 10 client.25477 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.393+0000 7f24b8f14b80 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.393+0000 7f24b8f14b80 20 client.25477 populate_metadata read hostname 'smithi082' 2023-07-14T19:32:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.393+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc15272d0 0x563bc15296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.393+0000 7f24b8f14b80 4 client.25477 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:32:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.393+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bc14dcd40 con 0x563bc15272d0 2023-07-14T19:32:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.401+0000 7f24a659c700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc15272d0 0x563bc15296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.419+0000 7f24a659c700 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc15272d0 0x563bc15296e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c9795a1356c6f2 server_cookie=22e221fc4cc26dec in_seq=0 out_seq=0 2023-07-14T19:32:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.419+0000 7f248dffb700 10 client.25477 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.430+0000 7f248dffb700 1 -- 172.21.15.82:0/639103217 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563bc14dcd40 con 0x563bc15272d0 2023-07-14T19:32:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.430+0000 7f248dffb700 10 client.25477 handle_command_reply: tid=0 2023-07-14T19:32:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.430+0000 7f24b8f14b80 1 client.25477 shutdown 2023-07-14T19:32:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 msgr2=0x563bc151d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f24a00a19a0 tx=0x7f24a009af70 comp rx=0 tx=0).stop 2023-07-14T19:32:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 shutdown_connections 2023-07-14T19:32:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc151ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc15272d0 0x563bc15296e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/639103217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.441+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 >> 172.21.15.82:0/639103217 conn(0x563bc15033d0 msgr2=0x563bc1507c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.442+0000 7f248dffb700 0 client.25477 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.442+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 shutdown_connections 2023-07-14T19:32:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.442+0000 7f24b8f14b80 1 -- 172.21.15.82:0/639103217 wait complete. 2023-07-14T19:32:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.442+0000 7f24b8f14b80 20 client.25477 trim_cache size 0 max 16384 2023-07-14T19:32:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.510+0000 7f24b8f14b80 1 Processor -- start 2023-07-14T19:32:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.533+0000 7f24b8f14b80 1 -- start start 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc1523bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc13b8490 con 0x563bc1513a90 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc150e850 con 0x563bc1517050 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc1509510 con 0x563bc1513f00 2023-07-14T19:32:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24a5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53226/0 (socket says 172.21.15.82:53226) 2023-07-14T19:32:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24a5d9b700 1 -- 172.21.15.82:0/2999431944 learned_addr learned my addr 172.21.15.82:0/2999431944 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.534+0000 7f24a6d9d700 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.535+0000 7f24a5d9b700 1 -- 172.21.15.82:0/2999431944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 msgr2=0x563bc15129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.535+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.535+0000 7f24a5d9b700 1 -- 172.21.15.82:0/2999431944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 msgr2=0x563bc1523bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.535+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc1523bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.535+0000 7f24a5d9b700 1 -- 172.21.15.82:0/2999431944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bc12b31f0 con 0x563bc1513f00 2023-07-14T19:32:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.535+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f24a00cc1d0 tx=0x7f24a000efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62686ae5e120891 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:34.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.631+0000 7f24957fa700 1 -- 172.21.15.82:0/2999431944 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0095900 con 0x563bc1513f00 2023-07-14T19:32:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.631+0000 7f24957fa700 1 -- 172.21.15.82:0/2999431944 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a00b2260 con 0x563bc1513f00 2023-07-14T19:32:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.631+0000 7f24957fa700 1 -- 172.21.15.82:0/2999431944 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0095ac0 con 0x563bc1513f00 2023-07-14T19:32:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.632+0000 7f24b8f14b80 1 -- 172.21.15.82:0/2999431944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 msgr2=0x563bc151d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.632+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f24a00cc1d0 tx=0x7f24a000efa0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.643+0000 7f24b8f14b80 1 -- 172.21.15.82:0/2999431944 shutdown_connections 2023-07-14T19:32:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.643+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1517050 0x563bc15129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.643+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1513a90 0x563bc1523bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.643+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/2999431944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513f00 0x563bc151d270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.643+0000 7f24b8f14b80 1 -- 172.21.15.82:0/2999431944 >> 172.21.15.82:0/2999431944 conn(0x563bc15033d0 msgr2=0x563bc1514300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:34.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.678+0000 7f24b8f14b80 1 -- 172.21.15.82:0/2999431944 shutdown_connections 2023-07-14T19:32:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.678+0000 7f24b8f14b80 1 -- 172.21.15.82:0/2999431944 wait complete. 2023-07-14T19:32:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.679+0000 7f24b8f14b80 1 -- start start 2023-07-14T19:32:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.679+0000 7f24b8f14b80 1 Processor -- start 2023-07-14T19:32:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513a90 0x563bc151fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513f00 0x563bc15200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1517050 0x563bc1520600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc1579f20 con 0x563bc1517050 2023-07-14T19:32:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc157a060 con 0x563bc1513f00 2023-07-14T19:32:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc157a1d0 con 0x563bc1513a90 2023-07-14T19:32:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.680+0000 7f24b8f14b80 10 client.0 _ll_register_callbacks cb 0x7f24a8069ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:32:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.681+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1517050 0x563bc1520600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.681+0000 7f24a6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1517050 0x563bc1520600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43970/0 (socket says 172.21.15.82:43970) 2023-07-14T19:32:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.681+0000 7f24a6d9d700 1 -- 172.21.15.82:0/866192435 learned_addr learned my addr 172.21.15.82:0/866192435 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.683+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513f00 0x563bc15200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.683+0000 7f24a659c700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513a90 0x563bc151fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.684+0000 7f24a5d9b700 1 -- 172.21.15.82:0/866192435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513a90 msgr2=0x563bc151fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.684+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513a90 0x563bc151fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.684+0000 7f24a5d9b700 1 -- 172.21.15.82:0/866192435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1517050 msgr2=0x563bc1520600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.684+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1517050 0x563bc1520600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.684+0000 7f24a5d9b700 1 -- 172.21.15.82:0/866192435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bc1275c00 con 0x563bc1513f00 2023-07-14T19:32:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.705+0000 7f24a5d9b700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513f00 0x563bc15200c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a00048d0 tx=0x7f24a00cc0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaec209b3cf37a51 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.841+0000 7f248dffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:32:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.841+0000 7f248dffb700 1 -- 172.21.15.82:0/866192435 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0095900 con 0x563bc1513f00 2023-07-14T19:32:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.841+0000 7f248dffb700 1 -- 172.21.15.82:0/866192435 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f24a0099b10 con 0x563bc1513f00 2023-07-14T19:32:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.841+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24a009db40 con 0x563bc1513f00 2023-07-14T19:32:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.845+0000 7f248dffb700 1 -- 172.21.15.82:0/866192435 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a0099650 con 0x563bc1513f00 2023-07-14T19:32:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.845+0000 7f24b8f14b80 10 client.25646 fetch_fsmap learned FSMap version 36 2023-07-14T19:32:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.845+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563bc128b250 con 0x563bc1513f00 2023-07-14T19:32:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.850+0000 7f248dffb700 1 -- 172.21.15.82:0/866192435 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a009db40 con 0x563bc1513f00 2023-07-14T19:32:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.850+0000 7f248dffb700 1 -- 172.21.15.82:0/866192435 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f24a00c0070 con 0x563bc1513f00 2023-07-14T19:32:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.851+0000 7f24b8f14b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:32:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.851+0000 7f24b8f14b80 10 client.25646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:32:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.851+0000 7f24b8f14b80 20 client.25646 populate_metadata read hostname 'smithi082' 2023-07-14T19:32:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.851+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc157b3f0 0x563bc157d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.851+0000 7f24b8f14b80 4 client.25646 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:32:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.851+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563bc1457e50 con 0x563bc157b3f0 2023-07-14T19:32:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.852+0000 7f24a659c700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc157b3f0 0x563bc157d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.853+0000 7f24a659c700 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc157b3f0 0x563bc157d7d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820e211bd308f565 server_cookie=46bb73585e36ae96 in_seq=0 out_seq=0 2023-07-14T19:32:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:34.866+0000 7f248dffb700 10 client.25646 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:35.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:35 smithi133 ceph-mon[128894]: pgmap v124: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-07-14T19:32:35.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:35 smithi082 ceph-mon[127461]: pgmap v124: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-07-14T19:32:35.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:35 smithi161 ceph-mon[129184]: pgmap v124: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2023-07-14T19:32:37.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:37 smithi133 ceph-mon[128894]: pgmap v125: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2023-07-14T19:32:37.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:37 smithi161 ceph-mon[129184]: pgmap v125: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2023-07-14T19:32:37.516 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:37 smithi082 ceph-mon[127461]: pgmap v125: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2023-07-14T19:32:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.938+0000 7f248dffb700 1 -- 172.21.15.82:0/866192435 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563bc1457e50 con 0x563bc157b3f0 2023-07-14T19:32:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.938+0000 7f248dffb700 10 client.25646 handle_command_reply: tid=0 2023-07-14T19:32:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.938+0000 7f24b8f14b80 1 client.25646 shutdown 2023-07-14T19:32:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513f00 msgr2=0x563bc15200c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513f00 0x563bc15200c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24a00048d0 tx=0x7f24a00cc0f0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 shutdown_connections 2023-07-14T19:32:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bc1517050 0x563bc1520600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563bc157b3f0 0x563bc157d7d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bc1513a90 0x563bc151fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 --2- 172.21.15.82:0/866192435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bc1513f00 0x563bc15200c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 >> 172.21.15.82:0/866192435 conn(0x563bc15033d0 msgr2=0x563bc1514300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.939+0000 7f248dffb700 0 client.25646 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.940+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 shutdown_connections 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.940+0000 7f24b8f14b80 1 -- 172.21.15.82:0/866192435 wait complete. 2023-07-14T19:32:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.940+0000 7f24b8f14b80 20 client.25646 trim_cache size 0 max 16384 2023-07-14T19:32:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:32:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:32:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:32:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:32:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f24900a14d0 msgr2=0x7f24900a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f24900a14d0 0x7f24900a3980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x563bc15142d0 tx=0x7f249c0047c0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 msgr2=0x7f24a81ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 0x7f24a81ab1c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e3c0 tx=0x7f24a000e780 comp rx=0 tx=0).stop 2023-07-14T19:32:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 shutdown_connections 2023-07-14T19:32:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f24900a14d0 0x7f24900a3980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a810e3d0 0x7f24a81a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a806ca40 0x7f24a81aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 --2- 172.21.15.82:0/511942913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a806d4d0 0x7f24a81ab1c0 secure :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f24a000e3c0 tx=0x7f24a000e780 comp rx=0 tx=0).stop 2023-07-14T19:32:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.959+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 >> 172.21.15.82:0/511942913 conn(0x7f24a806bea0 msgr2=0x7f24a80708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.962+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 shutdown_connections 2023-07-14T19:32:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:38.962+0000 7f24acd7c700 1 -- 172.21.15.82:0/511942913 wait complete. 2023-07-14T19:32:38.991 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:38 smithi161 ceph-mon[129184]: pgmap v126: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 136 op/s 2023-07-14T19:32:39.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:38 smithi133 ceph-mon[128894]: pgmap v126: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 136 op/s 2023-07-14T19:32:39.240 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:38 smithi082 ceph-mon[127461]: pgmap v126: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 136 op/s 2023-07-14T19:32:41.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:40 smithi133 ceph-mon[128894]: pgmap v127: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 428 op/s 2023-07-14T19:32:41.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:40 smithi082 ceph-mon[127461]: pgmap v127: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 428 op/s 2023-07-14T19:32:41.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:40 smithi161 ceph-mon[129184]: pgmap v127: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 428 op/s 2023-07-14T19:32:42.191 INFO:tasks.workunit.client.0.smithi082.stdout: 270 2563976 505 1813795 472 911261 1135 2023-07-14T19:32:43.185 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:42 smithi133 ceph-mon[128894]: pgmap v128: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 832 op/s 2023-07-14T19:32:43.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:42 smithi082 ceph-mon[127461]: pgmap v128: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 832 op/s 2023-07-14T19:32:43.242 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:42 smithi161 ceph-mon[129184]: pgmap v128: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 832 op/s 2023-07-14T19:32:45.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:44 smithi133 ceph-mon[128894]: pgmap v129: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 831 op/s 2023-07-14T19:32:45.223 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:44 smithi082 ceph-mon[127461]: pgmap v129: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 831 op/s 2023-07-14T19:32:45.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:44 smithi161 ceph-mon[129184]: pgmap v129: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 831 op/s 2023-07-14T19:32:47.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:46 smithi133 ceph-mon[128894]: pgmap v130: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.21k op/s 2023-07-14T19:32:47.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:46 smithi082 ceph-mon[127461]: pgmap v130: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.21k op/s 2023-07-14T19:32:47.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:46 smithi161 ceph-mon[129184]: pgmap v130: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 1.21k op/s 2023-07-14T19:32:49.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:48 smithi133 ceph-mon[128894]: pgmap v131: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.9 MiB/s wr, 1.21k op/s 2023-07-14T19:32:49.227 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:48 smithi082 ceph-mon[127461]: pgmap v131: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.9 MiB/s wr, 1.21k op/s 2023-07-14T19:32:49.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:48 smithi161 ceph-mon[129184]: pgmap v131: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.9 MiB/s wr, 1.21k op/s 2023-07-14T19:32:51.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:50 smithi082 ceph-mon[127461]: pgmap v132: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2023-07-14T19:32:51.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:50 smithi161 ceph-mon[129184]: pgmap v132: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2023-07-14T19:32:51.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:50 smithi133 ceph-mon[128894]: pgmap v132: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2023-07-14T19:32:52.200 INFO:tasks.workunit.client.0.smithi082.stdout: 277 2194129 553 1550338 378 807192 968 2023-07-14T19:32:53.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:52 smithi161 ceph-mon[129184]: pgmap v133: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 1.23k op/s 2023-07-14T19:32:53.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:52 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:32:53.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:52 smithi133 ceph-mon[128894]: pgmap v133: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 1.23k op/s 2023-07-14T19:32:53.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:52 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:32:53.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:52 smithi082 ceph-mon[127461]: pgmap v133: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 12 MiB/s wr, 1.23k op/s 2023-07-14T19:32:53.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:52 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:32:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.746+0000 7fce82f0d700 1 -- 172.21.15.82:0/513818795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 msgr2=0x7fce7c10e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.746+0000 7fce82f0d700 1 --2- 172.21.15.82:0/513818795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 0x7fce7c10e570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce78009990 tx=0x7fce78009ca0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 -- 172.21.15.82:0/513818795 shutdown_connections 2023-07-14T19:32:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 --2- 172.21.15.82:0/513818795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7c06e640 0x7fce7c11b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 --2- 172.21.15.82:0/513818795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 0x7fce7c10e570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 --2- 172.21.15.82:0/513818795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06dc90 0x7fce7c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 -- 172.21.15.82:0/513818795 >> 172.21.15.82:0/513818795 conn(0x7fce7c071000 msgr2=0x7fce7c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:53.749 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 -- 172.21.15.82:0/513818795 shutdown_connections 2023-07-14T19:32:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.747+0000 7fce82f0d700 1 -- 172.21.15.82:0/513818795 wait complete. 2023-07-14T19:32:53.750 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.748+0000 7fce82f0d700 1 Processor -- start 2023-07-14T19:32:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.748+0000 7fce82f0d700 1 -- start start 2023-07-14T19:32:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce82f0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7c06dc90 0x7fce7c1aa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce82f0d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 0x7fce7c1aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce82f0d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 0x7fce7c1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce82f0d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1a4f50 con 0x7fce7c06dc90 2023-07-14T19:32:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce82f0d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1a50c0 con 0x7fce7c06e640 2023-07-14T19:32:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce82f0d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1a5280 con 0x7fce7c10e1a0 2023-07-14T19:32:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.749+0000 7fce8270c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 0x7fce7c1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 0x7fce7c1aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce81f0b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7c06dc90 0x7fce7c1aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 0x7fce7c1aad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40328/0 (socket says 172.21.15.82:40328) 2023-07-14T19:32:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 -- 172.21.15.82:0/1883483254 learned_addr learned my addr 172.21.15.82:0/1883483254 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 -- 172.21.15.82:0/1883483254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 msgr2=0x7fce7c1a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 0x7fce7c1a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 -- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7c06dc90 msgr2=0x7fce7c1aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7c06dc90 0x7fce7c1aa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 -- 172.21.15.82:0/1883483254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780095f0 con 0x7fce7c06e640 2023-07-14T19:32:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.750+0000 7fce8170a700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 0x7fce7c1aad60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e9a0 tx=0x7fce6c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:53.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.822+0000 7fce72ffd700 1 -- 172.21.15.82:0/1883483254 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009790 con 0x7fce7c06e640 2023-07-14T19:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.822+0000 7fce72ffd700 1 -- 172.21.15.82:0/1883483254 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c00cbd0 con 0x7fce7c06e640 2023-07-14T19:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.822+0000 7fce72ffd700 1 -- 172.21.15.82:0/1883483254 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c018680 con 0x7fce7c06e640 2023-07-14T19:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.822+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1a5560 con 0x7fce7c06e640 2023-07-14T19:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.822+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c10ef50 con 0x7fce7c06e640 2023-07-14T19:32:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.829+0000 7fce90307b80 1 Processor -- start 2023-07-14T19:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.830+0000 7fce90307b80 1 -- start start 2023-07-14T19:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce90307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce90307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce90307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce90307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc663a6c50 con 0x55fc664a2170 2023-07-14T19:32:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce90307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc663f99e0 con 0x55fc664a7cc0 2023-07-14T19:32:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce90307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc663f2e70 con 0x55fc662fd6b0 2023-07-14T19:32:53.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce8270c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce8270c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53252/0 (socket says 172.21.15.82:53252) 2023-07-14T19:32:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce8270c700 1 -- 172.21.15.82:0/843498113 learned_addr learned my addr 172.21.15.82:0/843498113 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:53.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.831+0000 7fce81f0b700 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.832+0000 7fce8270c700 1 -- 172.21.15.82:0/843498113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 msgr2=0x55fc664a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.832+0000 7fce8270c700 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.837 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.832+0000 7fce8270c700 1 -- 172.21.15.82:0/843498113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 msgr2=0x55fc664a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.832+0000 7fce8270c700 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.832+0000 7fce8270c700 1 -- 172.21.15.82:0/843498113 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc66220d80 con 0x55fc662fd6b0 2023-07-14T19:32:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.832+0000 7fce8270c700 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664a7780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce74003f40 tx=0x7fce74009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ba81699824b9b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.843+0000 7fce72ffd700 1 -- 172.21.15.82:0/1883483254 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fce6c00cd40 con 0x7fce7c06e640 2023-07-14T19:32:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.843+0000 7fce67fff700 1 -- 172.21.15.82:0/843498113 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74019070 con 0x55fc662fd6b0 2023-07-14T19:32:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.843+0000 7fce67fff700 1 -- 172.21.15.82:0/843498113 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce7400f040 con 0x55fc662fd6b0 2023-07-14T19:32:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.843+0000 7fce67fff700 1 -- 172.21.15.82:0/843498113 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74007ae0 con 0x55fc662fd6b0 2023-07-14T19:32:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.844+0000 7fce90307b80 1 -- 172.21.15.82:0/843498113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 msgr2=0x55fc664a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.844+0000 7fce90307b80 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664a7780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fce74003f40 tx=0x7fce74009ba0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.845+0000 7fce72ffd700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fce680a14d0 0x7fce680a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.845+0000 7fce72ffd700 1 -- 172.21.15.82:0/1883483254 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fce6c014070 con 0x7fce7c06e640 2023-07-14T19:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.850+0000 7fce8270c700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fce680a14d0 0x7fce680a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.869+0000 7fce8270c700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fce680a14d0 0x7fce680a3980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce74003f40 tx=0x7fce74007c80 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.910+0000 7fce90307b80 1 -- 172.21.15.82:0/843498113 shutdown_connections 2023-07-14T19:32:53.911 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.910+0000 7fce90307b80 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.910+0000 7fce90307b80 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664a7780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.910+0000 7fce90307b80 1 --2- 172.21.15.82:0/843498113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.910+0000 7fce90307b80 1 -- 172.21.15.82:0/843498113 >> 172.21.15.82:0/843498113 conn(0x55fc66497620 msgr2=0x55fc6649c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.911+0000 7fce90307b80 1 -- 172.21.15.82:0/843498113 shutdown_connections 2023-07-14T19:32:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.911+0000 7fce90307b80 1 -- 172.21.15.82:0/843498113 wait complete. 2023-07-14T19:32:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.911+0000 7fce90307b80 1 -- start start 2023-07-14T19:32:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.911+0000 7fce90307b80 1 Processor -- start 2023-07-14T19:32:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc66392a60 con 0x55fc664a2170 2023-07-14T19:32:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc66392760 con 0x55fc664a7cc0 2023-07-14T19:32:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc662768c0 con 0x55fc662fd6b0 2023-07-14T19:32:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.912+0000 7fce90307b80 10 client.0 _ll_register_callbacks cb 0x55fc66390700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:32:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.917+0000 7fce8170a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.917+0000 7fce8170a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664ab1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43994/0 (socket says 172.21.15.82:43994) 2023-07-14T19:32:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.917+0000 7fce8170a700 1 -- 172.21.15.82:0/3219966310 learned_addr learned my addr 172.21.15.82:0/3219966310 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.918+0000 7fce8270c700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.918+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.919+0000 7fce81f0b700 1 -- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 msgr2=0x55fc664ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.919+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.919+0000 7fce81f0b700 1 -- 172.21.15.82:0/3219966310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 msgr2=0x55fc664ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.919+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.919+0000 7fce81f0b700 1 -- 172.21.15.82:0/3219966310 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce74003b70 con 0x55fc662fd6b0 2023-07-14T19:32:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.924+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664adf30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce7800b2e0 tx=0x7fce7800bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e791ac747b84244 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.973+0000 7fce657fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:32:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.973+0000 7fce657fa700 1 -- 172.21.15.82:0/3219966310 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78020040 con 0x55fc662fd6b0 2023-07-14T19:32:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.973+0000 7fce657fa700 1 -- 172.21.15.82:0/3219966310 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fce78004500 con 0x55fc662fd6b0 2023-07-14T19:32:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.982+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc6634c490 con 0x55fc662fd6b0 2023-07-14T19:32:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.987+0000 7fce657fa700 1 -- 172.21.15.82:0/3219966310 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce780046c0 con 0x55fc662fd6b0 2023-07-14T19:32:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.999+0000 7fce90307b80 10 client.25495 fetch_fsmap learned FSMap version 36 2023-07-14T19:32:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:53.999+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fc66209c00 con 0x55fc662fd6b0 2023-07-14T19:32:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.002+0000 7fce657fa700 1 -- 172.21.15.82:0/3219966310 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc6634c490 con 0x55fc662fd6b0 2023-07-14T19:32:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.002+0000 7fce657fa700 1 -- 172.21.15.82:0/3219966310 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fce7802f830 con 0x55fc662fd6b0 2023-07-14T19:32:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.004+0000 7fce90307b80 10 client.25495 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:32:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.004+0000 7fce90307b80 10 client.25495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:32:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.004+0000 7fce90307b80 20 client.25495 populate_metadata read hostname 'smithi082' 2023-07-14T19:32:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.004+0000 7fce90307b80 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc664bb520 0x55fc664bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.004+0000 7fce90307b80 4 client.25495 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:32:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.004+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc66470d40 con 0x55fc664bb520 2023-07-14T19:32:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.012+0000 7fce8170a700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc664bb520 0x55fc664bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.022+0000 7fce657fa700 10 client.25495 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.022+0000 7fce8170a700 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc664bb520 0x55fc664bd930 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ba6077f13fb0d7 server_cookie=912d554c29c2ec6d in_seq=0 out_seq=0 2023-07-14T19:32:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.023+0000 7fce657fa700 1 -- 172.21.15.82:0/3219966310 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55fc66470d40 con 0x55fc664bb520 2023-07-14T19:32:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.023+0000 7fce657fa700 10 client.25495 handle_command_reply: tid=0 2023-07-14T19:32:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.023+0000 7fce90307b80 1 client.25495 shutdown 2023-07-14T19:32:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 msgr2=0x55fc664adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664adf30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce7800b2e0 tx=0x7fce7800bf30 comp rx=0 tx=0).stop 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 shutdown_connections 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc664bb520 0x55fc664bd930 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc662fd6b0 0x55fc664adf30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 --2- 172.21.15.82:0/3219966310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a2170 0x55fc664ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 >> 172.21.15.82:0/3219966310 conn(0x55fc66497620 msgr2=0x55fc6649be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.025+0000 7fce657fa700 0 client.25495 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.026+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 shutdown_connections 2023-07-14T19:32:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.026+0000 7fce90307b80 1 -- 172.21.15.82:0/3219966310 wait complete. 2023-07-14T19:32:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.026+0000 7fce90307b80 20 client.25495 trim_cache size 0 max 16384 2023-07-14T19:32:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.105+0000 7fce90307b80 1 Processor -- start 2023-07-14T19:32:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.108+0000 7fce90307b80 1 -- start start 2023-07-14T19:32:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce90307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc662fd6b0 0x55fc664b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce90307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc6649c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce90307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce90307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc664a7780 con 0x55fc662fd6b0 2023-07-14T19:32:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce90307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc664ac790 con 0x55fc664a7cc0 2023-07-14T19:32:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce90307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc664c2730 con 0x55fc664a2170 2023-07-14T19:32:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce8270c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce8170a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc6649c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce8170a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc6649c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53268/0 (socket says 172.21.15.82:53268) 2023-07-14T19:32:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.109+0000 7fce8170a700 1 -- 172.21.15.82:0/3941631045 learned_addr learned my addr 172.21.15.82:0/3941631045 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.110+0000 7fce8170a700 1 -- 172.21.15.82:0/3941631045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 msgr2=0x55fc664a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.110+0000 7fce8170a700 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.110+0000 7fce8170a700 1 -- 172.21.15.82:0/3941631045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc662fd6b0 msgr2=0x55fc664b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.110+0000 7fce8170a700 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc662fd6b0 0x55fc664b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.110+0000 7fce8170a700 1 -- 172.21.15.82:0/3941631045 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc662471f0 con 0x55fc664a2170 2023-07-14T19:32:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.118+0000 7fce8170a700 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc6649c600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fce6c09c190 tx=0x7fce6c0cf1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9daa62efc3e3d6a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.132+0000 7fce70ff9700 1 -- 172.21.15.82:0/3941631045 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c09c250 con 0x55fc664a2170 2023-07-14T19:32:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.132+0000 7fce70ff9700 1 -- 172.21.15.82:0/3941631045 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c0a0280 con 0x55fc664a2170 2023-07-14T19:32:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.132+0000 7fce70ff9700 1 -- 172.21.15.82:0/3941631045 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c09c710 con 0x55fc664a2170 2023-07-14T19:32:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.133+0000 7fce90307b80 1 -- 172.21.15.82:0/3941631045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 msgr2=0x55fc6649c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.133+0000 7fce90307b80 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc6649c600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fce6c09c190 tx=0x7fce6c0cf1a0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.175+0000 7fce90307b80 1 -- 172.21.15.82:0/3941631045 shutdown_connections 2023-07-14T19:32:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.175+0000 7fce90307b80 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc664a7cc0 0x55fc664a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.175+0000 7fce90307b80 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc662fd6b0 0x55fc664b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.175+0000 7fce90307b80 1 --2- 172.21.15.82:0/3941631045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc6649c600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.175+0000 7fce90307b80 1 -- 172.21.15.82:0/3941631045 >> 172.21.15.82:0/3941631045 conn(0x55fc66497620 msgr2=0x55fc664a8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.180+0000 7fce90307b80 1 -- 172.21.15.82:0/3941631045 shutdown_connections 2023-07-14T19:32:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.181+0000 7fce90307b80 1 -- 172.21.15.82:0/3941631045 wait complete. 2023-07-14T19:32:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 -- start start 2023-07-14T19:32:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 Processor -- start 2023-07-14T19:32:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc662fd6b0 0x55fc664aae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc664a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 0x55fc664a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc664aa3e0 con 0x55fc664a7cc0 2023-07-14T19:32:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc664aa550 con 0x55fc662fd6b0 2023-07-14T19:32:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.182+0000 7fce90307b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc664aa710 con 0x55fc664a2170 2023-07-14T19:32:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce90307b80 10 client.0 _ll_register_callbacks cb 0x7fce7800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:32:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 0x55fc664a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 0x55fc664a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44008/0 (socket says 172.21.15.82:44008) 2023-07-14T19:32:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 -- 172.21.15.82:0/3669511031 learned_addr learned my addr 172.21.15.82:0/3669511031 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:32:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc662fd6b0 0x55fc664aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8170a700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc664a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 -- 172.21.15.82:0/3669511031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 msgr2=0x55fc664a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc664a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 -- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc662fd6b0 msgr2=0x55fc664aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:32:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc662fd6b0 0x55fc664aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.183+0000 7fce8270c700 1 -- 172.21.15.82:0/3669511031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c10ef50 con 0x55fc664a7cc0 2023-07-14T19:32:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.184+0000 7fce8270c700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 0x55fc664a9ea0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fce74017040 tx=0x7fce74009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a492d0613314950a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:32:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.188+0000 7fce657fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:32:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.188+0000 7fce657fa700 1 -- 172.21.15.82:0/3669511031 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74019040 con 0x55fc664a7cc0 2023-07-14T19:32:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.188+0000 7fce657fa700 1 -- 172.21.15.82:0/3669511031 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fce74027070 con 0x55fc664a7cc0 2023-07-14T19:32:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.191+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce7400f040 con 0x55fc664a7cc0 2023-07-14T19:32:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.191+0000 7fce657fa700 1 -- 172.21.15.82:0/3669511031 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce740155a0 con 0x55fc664a7cc0 2023-07-14T19:32:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.193+0000 7fce90307b80 10 client.15963 fetch_fsmap learned FSMap version 36 2023-07-14T19:32:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.193+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fc6621f250 con 0x55fc664a7cc0 2023-07-14T19:32:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.193+0000 7fce657fa700 1 -- 172.21.15.82:0/3669511031 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce7400f040 con 0x55fc664a7cc0 2023-07-14T19:32:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.194+0000 7fce657fa700 1 -- 172.21.15.82:0/3669511031 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fce740368c0 con 0x55fc664a7cc0 2023-07-14T19:32:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.195+0000 7fce90307b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:32:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.195+0000 7fce90307b80 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:32:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.195+0000 7fce90307b80 20 client.15963 populate_metadata read hostname 'smithi082' 2023-07-14T19:32:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.195+0000 7fce90307b80 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc66510550 0x55fc66512960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:32:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.195+0000 7fce90307b80 4 client.15963 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:32:54.197 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.195+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc663ebe50 con 0x55fc66510550 2023-07-14T19:32:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.198+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc66510550 0x55fc66512960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:32:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.200+0000 7fce657fa700 10 client.15963 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:54.200+0000 7fce81f0b700 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc66510550 0x55fc66512960 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3183554e849e8a47 server_cookie=3e458d391760024c in_seq=0 out_seq=0 2023-07-14T19:32:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.406+0000 7fce657fa700 1 -- 172.21.15.82:0/3669511031 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc663ebe50 con 0x55fc66510550 2023-07-14T19:32:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.406+0000 7fce657fa700 10 client.15963 handle_command_reply: tid=0 2023-07-14T19:32:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.407+0000 7fce90307b80 1 client.15963 shutdown 2023-07-14T19:32:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 msgr2=0x55fc664a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 0x55fc664a9ea0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fce74017040 tx=0x7fce74009100 comp rx=0 tx=0).stop 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 shutdown_connections 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc664a7cc0 0x55fc664a9ea0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc662fd6b0 0x55fc664aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55fc66510550 0x55fc66512960 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 --2- 172.21.15.82:0/3669511031 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc664a2170 0x55fc664a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 >> 172.21.15.82:0/3669511031 conn(0x55fc66497620 msgr2=0x55fc664a8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.412+0000 7fce657fa700 0 client.15963 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:32:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.413+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 shutdown_connections 2023-07-14T19:32:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.413+0000 7fce90307b80 1 -- 172.21.15.82:0/3669511031 wait complete. 2023-07-14T19:32:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.413+0000 7fce90307b80 20 client.15963 trim_cache size 0 max 16384 2023-07-14T19:32:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:32:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:32:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:32:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:32:55.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:55 smithi133 ceph-mon[128894]: pgmap v134: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 830 op/s 2023-07-14T19:32:55.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:55 smithi082 ceph-mon[127461]: pgmap v134: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 830 op/s 2023-07-14T19:32:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.472+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fce680a14d0 msgr2=0x7fce680a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.472+0000 7fce82f0d700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fce680a14d0 0x7fce680a3980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fce74003f40 tx=0x7fce74007c80 comp rx=0 tx=0).stop 2023-07-14T19:32:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.472+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 msgr2=0x7fce7c1aad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:32:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.472+0000 7fce82f0d700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 0x7fce7c1aad60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00e9a0 tx=0x7fce6c00ecb0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.489+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 shutdown_connections 2023-07-14T19:32:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.489+0000 7fce82f0d700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fce680a14d0 0x7fce680a3980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.489+0000 7fce82f0d700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce7c10e1a0 0x7fce7c1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.489+0000 7fce82f0d700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce7c06dc90 0x7fce7c1aa820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.489+0000 7fce82f0d700 1 --2- 172.21.15.82:0/1883483254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce7c06e640 0x7fce7c1aad60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:32:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.489+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 >> 172.21.15.82:0/1883483254 conn(0x7fce7c071000 msgr2=0x7fce7c06acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:32:55.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:55 smithi161 ceph-mon[129184]: pgmap v134: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 11 MiB/s wr, 830 op/s 2023-07-14T19:32:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.490+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 shutdown_connections 2023-07-14T19:32:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:32:55.490+0000 7fce82f0d700 1 -- 172.21.15.82:0/1883483254 wait complete. 2023-07-14T19:32:57.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:57 smithi133 ceph-mon[128894]: pgmap v135: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-07-14T19:32:57.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:57 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:32:57.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:57 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:32:57.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:57 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:32:57.467 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:57 smithi082 ceph-mon[127461]: pgmap v135: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-07-14T19:32:57.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:57 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:32:57.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:57 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:32:57.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:57 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:32:57.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:57 smithi161 ceph-mon[129184]: pgmap v135: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-07-14T19:32:57.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:57 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:32:57.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:57 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:32:57.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:57 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:32:59.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:32:59 smithi133 ceph-mon[128894]: pgmap v136: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 787 op/s 2023-07-14T19:32:59.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:32:59 smithi082 ceph-mon[127461]: pgmap v136: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 787 op/s 2023-07-14T19:32:59.497 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:32:59 smithi161 ceph-mon[129184]: pgmap v136: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 787 op/s 2023-07-14T19:33:01.464 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:01 smithi082 ceph-mon[127461]: pgmap v137: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 876 op/s 2023-07-14T19:33:01.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:01 smithi161 ceph-mon[129184]: pgmap v137: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 876 op/s 2023-07-14T19:33:01.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:01 smithi133 ceph-mon[128894]: pgmap v137: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 876 op/s 2023-07-14T19:33:02.204 INFO:tasks.workunit.client.0.smithi082.stdout: 286 2569657 440 1807842 441 966267 1055 2023-07-14T19:33:03.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:03 smithi133 ceph-mon[128894]: pgmap v138: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 786 op/s 2023-07-14T19:33:03.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:03 smithi082 ceph-mon[127461]: pgmap v138: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 786 op/s 2023-07-14T19:33:03.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:03 smithi161 ceph-mon[129184]: pgmap v138: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 786 op/s 2023-07-14T19:33:05.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:05 smithi133 ceph-mon[128894]: pgmap v139: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 600 op/s 2023-07-14T19:33:05.713 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:05 smithi082 ceph-mon[127461]: pgmap v139: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 600 op/s 2023-07-14T19:33:05.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:05 smithi161 ceph-mon[129184]: pgmap v139: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 600 op/s 2023-07-14T19:33:06.683 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:06 smithi133 ceph-mon[128894]: pgmap v140: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 640 op/s 2023-07-14T19:33:06.719 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:06 smithi082 ceph-mon[127461]: pgmap v140: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 640 op/s 2023-07-14T19:33:06.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:06 smithi161 ceph-mon[129184]: pgmap v140: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 640 op/s 2023-07-14T19:33:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.909+0000 7f2be24b7700 1 -- 172.21.15.82:0/1128299850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06ca40 msgr2=0x7f2bdc06ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.909+0000 7f2be24b7700 1 --2- 172.21.15.82:0/1128299850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06ca40 0x7f2bdc06ceb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009990 tx=0x7f2bcc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 -- 172.21.15.82:0/1128299850 shutdown_connections 2023-07-14T19:33:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 --2- 172.21.15.82:0/1128299850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc11b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 --2- 172.21.15.82:0/1128299850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc10e3d0 0x7f2bdc10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 --2- 172.21.15.82:0/1128299850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06ca40 0x7f2bdc06ceb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 -- 172.21.15.82:0/1128299850 >> 172.21.15.82:0/1128299850 conn(0x7f2bdc06bea0 msgr2=0x7f2bdc071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 -- 172.21.15.82:0/1128299850 shutdown_connections 2023-07-14T19:33:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.919+0000 7f2be24b7700 1 -- 172.21.15.82:0/1128299850 wait complete. 2023-07-14T19:33:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.920+0000 7f2be24b7700 1 Processor -- start 2023-07-14T19:33:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.920+0000 7f2be24b7700 1 -- start start 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2be24b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc06ca40 0x7f2bdc1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2be24b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc1a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2be24b7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc10e3d0 0x7f2bdc1a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2be24b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1a8dd0 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2be24b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1a8f40 con 0x7f2bdc06ca40 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2be24b7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1a9100 con 0x7f2bdc10e3d0 2023-07-14T19:33:08.923 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc1a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc1a5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44020/0 (socket says 172.21.15.82:44020) 2023-07-14T19:33:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 -- 172.21.15.82:0/3012104398 learned_addr learned my addr 172.21.15.82:0/3012104398 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc06ca40 0x7f2bdc1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 -- 172.21.15.82:0/3012104398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc10e3d0 msgr2=0x7f2bdc1a87d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T19:33:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc10e3d0 0x7f2bdc1a87d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 -- 172.21.15.82:0/3012104398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc06ca40 msgr2=0x7f2bdc1a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.921+0000 7f2bdb7fe700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc06ca40 0x7f2bdc1a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.922+0000 7f2bdb7fe700 1 -- 172.21.15.82:0/3012104398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0095f0 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.922+0000 7f2bdb7fe700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc1a5210 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005f50 tx=0x7f2bcc003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.925+0000 7f2bd97fa700 1 -- 172.21.15.82:0/3012104398 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc03f070 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.925+0000 7f2bd97fa700 1 -- 172.21.15.82:0/3012104398 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bcc00f460 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.925+0000 7f2bd97fa700 1 -- 172.21.15.82:0/3012104398 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc039660 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.935 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.925+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc1a9380 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.925+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc072870 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 Processor -- start 2023-07-14T19:33:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 -- start start 2023-07-14T19:33:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6efc7c50 con 0x561a6f0c8f00 2023-07-14T19:33:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f01a9e0 con 0x561a6ef1e6b0 2023-07-14T19:33:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.930+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f013e70 con 0x561a6f0c8a90 2023-07-14T19:33:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.931+0000 7f2bdb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.931+0000 7f2bdb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44028/0 (socket says 172.21.15.82:44028) 2023-07-14T19:33:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.931+0000 7f2bdb7fe700 1 -- 172.21.15.82:0/2938006348 learned_addr learned my addr 172.21.15.82:0/2938006348 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.932+0000 7f2bd97fa700 1 -- 172.21.15.82:0/3012104398 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2bcc00f840 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.932+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bdbfff700 1 -- 172.21.15.82:0/2938006348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 msgr2=0x561a6f0c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bdbfff700 1 -- 172.21.15.82:0/2938006348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 msgr2=0x561a6f0c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bdbfff700 1 -- 172.21.15.82:0/2938006348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6ee41d80 con 0x561a6f0c8a90 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bd97fa700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2bc40a14d0 0x7f2bc40a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.933+0000 7f2bd97fa700 1 -- 172.21.15.82:0/3012104398 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f2bcc0efdf0 con 0x7f2bdc06d4d0 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.934+0000 7f2be0a54700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2bc40a14d0 0x7f2bc40a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.935+0000 7f2be0a54700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2bc40a14d0 0x7f2bc40a3980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4001f30 tx=0x7f2bd4001610 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.937+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0c29e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b7e0 tx=0x7f2bd000baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a534fedfdfca3ec server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.943+0000 7f2bc27fc700 1 -- 172.21.15.82:0/2938006348 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00125f0 con 0x561a6f0c8a90 2023-07-14T19:33:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.943+0000 7f2bc27fc700 1 -- 172.21.15.82:0/2938006348 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd0012c10 con 0x561a6f0c8a90 2023-07-14T19:33:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.943+0000 7f2bc27fc700 1 -- 172.21.15.82:0/2938006348 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0013de0 con 0x561a6f0c8a90 2023-07-14T19:33:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.944+0000 7f2bee64fb80 1 -- 172.21.15.82:0/2938006348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 msgr2=0x561a6f0c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.944+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0c29e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b7e0 tx=0x7f2bd000baf0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.953+0000 7f2bee64fb80 1 -- 172.21.15.82:0/2938006348 shutdown_connections 2023-07-14T19:33:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.953+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.953+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.953+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/2938006348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0c29e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.953+0000 7f2bee64fb80 1 -- 172.21.15.82:0/2938006348 >> 172.21.15.82:0/2938006348 conn(0x561a6f0b83d0 msgr2=0x561a6f0bd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.967+0000 7f2bee64fb80 1 -- 172.21.15.82:0/2938006348 shutdown_connections 2023-07-14T19:33:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.967+0000 7f2bee64fb80 1 -- 172.21.15.82:0/2938006348 wait complete. 2023-07-14T19:33:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 -- start start 2023-07-14T19:33:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 Processor -- start 2023-07-14T19:33:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6ef1e6b0 0x561a6f0cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6f0c8a90 0x561a6f0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6efb3a60 con 0x561a6ef1e6b0 2023-07-14T19:33:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6efb3760 con 0x561a6f0c8a90 2023-07-14T19:33:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6ee978c0 con 0x561a6f0c8f00 2023-07-14T19:33:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.968+0000 7f2bee64fb80 10 client.0 _ll_register_callbacks cb 0x561a6efb1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:33:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.970+0000 7f2be0a54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.970+0000 7f2be0a54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53302/0 (socket says 172.21.15.82:53302) 2023-07-14T19:33:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.970+0000 7f2be0a54700 1 -- 172.21.15.82:0/1412960456 learned_addr learned my addr 172.21.15.82:0/1412960456 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.971+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6ef1e6b0 0x561a6f0cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.971+0000 7f2bdbfff700 1 -- 172.21.15.82:0/1412960456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 msgr2=0x561a6f0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.971+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.971+0000 7f2bdbfff700 1 -- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6f0c8a90 msgr2=0x561a6f0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.971+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6f0c8a90 0x561a6f0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.971+0000 7f2bdbfff700 1 -- 172.21.15.82:0/1412960456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd000e040 con 0x561a6ef1e6b0 2023-07-14T19:33:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.972+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6ef1e6b0 0x561a6f0cee10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b350 tx=0x7f2bd00128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538751ca63f9a050 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.972+0000 7f2bbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:33:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.973+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/1412960456 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0013bd0 con 0x561a6ef1e6b0 2023-07-14T19:33:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.973+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/1412960456 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f2bd0013d30 con 0x561a6ef1e6b0 2023-07-14T19:33:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.973+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a6ef6d490 con 0x561a6ef1e6b0 2023-07-14T19:33:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.977+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/1412960456 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd0004040 con 0x561a6ef1e6b0 2023-07-14T19:33:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.977+0000 7f2bee64fb80 10 client.15975 fetch_fsmap learned FSMap version 36 2023-07-14T19:33:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.977+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561a6ee2ac00 con 0x561a6ef1e6b0 2023-07-14T19:33:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/1412960456 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a6ef6d490 con 0x561a6ef1e6b0 2023-07-14T19:33:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/1412960456 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2bd0003b30 con 0x561a6ef1e6b0 2023-07-14T19:33:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bee64fb80 10 client.15975 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:33:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bee64fb80 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:33:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bee64fb80 20 client.15975 populate_metadata read hostname 'smithi082' 2023-07-14T19:33:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f0ed310 0x561a6f0ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bee64fb80 4 client.15975 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:33:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.980+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a6f091d40 con 0x561a6f0ed310 2023-07-14T19:33:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:08.985+0000 7f2bdb7fe700 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f0ed310 0x561a6f0ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.004+0000 7f2bdb7fe700 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f0ed310 0x561a6f0ef720 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed251fd08c06a928 server_cookie=97b70a05050bdb9c in_seq=0 out_seq=0 2023-07-14T19:33:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.004+0000 7f2bbb7fe700 10 client.15975 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.007+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/1412960456 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x561a6f091d40 con 0x561a6f0ed310 2023-07-14T19:33:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.007+0000 7f2bbb7fe700 10 client.15975 handle_command_reply: tid=0 2023-07-14T19:33:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.007+0000 7f2bee64fb80 1 client.15975 shutdown 2023-07-14T19:33:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6ef1e6b0 msgr2=0x561a6f0cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6ef1e6b0 0x561a6f0cee10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000b350 tx=0x7f2bd00128d0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 shutdown_connections 2023-07-14T19:33:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f0ed310 0x561a6f0ef720 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6ef1e6b0 0x561a6f0cee10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/1412960456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6f0c8a90 0x561a6f0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 >> 172.21.15.82:0/1412960456 conn(0x561a6f0b83d0 msgr2=0x561a6f0bcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bbb7fe700 0 client.15975 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 shutdown_connections 2023-07-14T19:33:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 1 -- 172.21.15.82:0/1412960456 wait complete. 2023-07-14T19:33:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.008+0000 7f2bee64fb80 20 client.15975 trim_cache size 0 max 16384 2023-07-14T19:33:09.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.045+0000 7f2bee64fb80 1 Processor -- start 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.045+0000 7f2bee64fb80 1 -- start start 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.046+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.046+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.046+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.046+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f0befc0 con 0x561a6f0c8f00 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.046+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f0ea9a0 con 0x561a6ef1e6b0 2023-07-14T19:33:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.046+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f0c29e0 con 0x561a6f0c8a90 2023-07-14T19:33:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44042/0 (socket says 172.21.15.82:44042) 2023-07-14T19:33:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 -- 172.21.15.82:0/152662767 learned_addr learned my addr 172.21.15.82:0/152662767 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 -- 172.21.15.82:0/152662767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 msgr2=0x561a6f0bd3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:33:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0bd3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 -- 172.21.15.82:0/152662767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 msgr2=0x561a6f0e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.047+0000 7f2be0a54700 1 -- 172.21.15.82:0/152662767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a6ee681f0 con 0x561a6f0c8f00 2023-07-14T19:33:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.048+0000 7f2be0a54700 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0cb850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400cdf0 tx=0x7f2bd400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9389006dc2a19f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.073+0000 7f2bc2ffd700 1 -- 172.21.15.82:0/152662767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4014dc0 con 0x561a6f0c8f00 2023-07-14T19:33:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.073+0000 7f2bc2ffd700 1 -- 172.21.15.82:0/152662767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd401c040 con 0x561a6f0c8f00 2023-07-14T19:33:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.073+0000 7f2bc2ffd700 1 -- 172.21.15.82:0/152662767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40203f0 con 0x561a6f0c8f00 2023-07-14T19:33:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.074+0000 7f2bee64fb80 1 -- 172.21.15.82:0/152662767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 msgr2=0x561a6f0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.074+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0cb850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400cdf0 tx=0x7f2bd400f000 comp rx=0 tx=0).stop 2023-07-14T19:33:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.080+0000 7f2bee64fb80 1 -- 172.21.15.82:0/152662767 shutdown_connections 2023-07-14T19:33:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.080+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8f00 0x561a6f0cb850 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.080+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.080+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/152662767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8a90 0x561a6f0bd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.080+0000 7f2bee64fb80 1 -- 172.21.15.82:0/152662767 >> 172.21.15.82:0/152662767 conn(0x561a6f0b83d0 msgr2=0x561a6f0e8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.095+0000 7f2bee64fb80 1 -- 172.21.15.82:0/152662767 shutdown_connections 2023-07-14T19:33:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.102+0000 7f2bee64fb80 1 -- 172.21.15.82:0/152662767 wait complete. 2023-07-14T19:33:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.102+0000 7f2bee64fb80 1 -- start start 2023-07-14T19:33:09.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 Processor -- start 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8a90 0x561a6f0c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f0c4670 con 0x561a6f0c8a90 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f0c47e0 con 0x561a6ef1e6b0 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a6f0c49a0 con 0x561a6f0c8f00 2023-07-14T19:33:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.103+0000 7f2bee64fb80 10 client.0 _ll_register_callbacks cb 0x7f2bcc0f3770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.105+0000 7f2be0a54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.105+0000 7f2be0a54700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53312/0 (socket says 172.21.15.82:53312) 2023-07-14T19:33:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.105+0000 7f2be0a54700 1 -- 172.21.15.82:0/3104772573 learned_addr learned my addr 172.21.15.82:0/3104772573 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.108+0000 7f2be0a54700 1 -- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 msgr2=0x561a6f0c5350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:33:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.108+0000 7f2be0a54700 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0c5350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.108+0000 7f2be0a54700 1 -- 172.21.15.82:0/3104772573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8a90 msgr2=0x561a6f0c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.108+0000 7f2be0a54700 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8a90 0x561a6f0c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.108+0000 7f2be0a54700 1 -- 172.21.15.82:0/3104772573 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd400cad0 con 0x561a6f0c8f00 2023-07-14T19:33:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.118+0000 7f2be0a54700 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0c3fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4004b10 tx=0x7f2bd4003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3318818bcf60ce2e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.118+0000 7f2bbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:33:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.124+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/3104772573 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4004380 con 0x561a6f0c8f00 2023-07-14T19:33:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.124+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/3104772573 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f2bd4014bc0 con 0x561a6f0c8f00 2023-07-14T19:33:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.124+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd0012c10 con 0x561a6f0c8f00 2023-07-14T19:33:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.135+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/3104772573 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bd4014d80 con 0x561a6f0c8f00 2023-07-14T19:33:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.136+0000 7f2bee64fb80 10 client.25510 fetch_fsmap learned FSMap version 36 2023-07-14T19:33:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.136+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561a6ee40250 con 0x561a6f0c8f00 2023-07-14T19:33:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.160+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/3104772573 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bd0012c10 con 0x561a6f0c8f00 2023-07-14T19:33:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.160+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/3104772573 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2bd4029540 con 0x561a6f0c8f00 2023-07-14T19:33:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.161+0000 7f2bee64fb80 10 client.25510 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:33:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.161+0000 7f2bee64fb80 10 client.25510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:33:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.161+0000 7f2bee64fb80 20 client.25510 populate_metadata read hostname 'smithi082' 2023-07-14T19:33:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.161+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f142060 0x561a6f144470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.161+0000 7f2bee64fb80 4 client.25510 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:33:09.164 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.161+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a6f00ce50 con 0x561a6f142060 2023-07-14T19:33:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.169+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f142060 0x561a6f144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:09.184 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:08 smithi133 ceph-mon[128894]: pgmap v141: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-07-14T19:33:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.189+0000 7f2bdbfff700 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f142060 0x561a6f144470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd58e56fd940627 server_cookie=239bb349f7c15937 in_seq=0 out_seq=0 2023-07-14T19:33:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:09.189+0000 7f2bbb7fe700 10 client.25510 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:09.214 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:08 smithi082 ceph-mon[127461]: pgmap v141: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-07-14T19:33:09.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:08 smithi161 ceph-mon[129184]: pgmap v141: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 346 op/s 2023-07-14T19:33:11.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:10 smithi133 ceph-mon[128894]: pgmap v142: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2023-07-14T19:33:11.213 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:10 smithi082 ceph-mon[127461]: pgmap v142: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2023-07-14T19:33:11.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:10 smithi161 ceph-mon[129184]: pgmap v142: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 453 op/s 2023-07-14T19:33:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.898+0000 7f2bbb7fe700 1 -- 172.21.15.82:0/3104772573 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a6f00ce50 con 0x561a6f142060 2023-07-14T19:33:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.898+0000 7f2bbb7fe700 10 client.25510 handle_command_reply: tid=0 2023-07-14T19:33:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.902+0000 7f2bee64fb80 1 client.25510 shutdown 2023-07-14T19:33:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.909+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 msgr2=0x561a6f0c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.909+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0c3fd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4004b10 tx=0x7f2bd4003ce0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 shutdown_connections 2023-07-14T19:33:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x561a6f142060 0x561a6f144470 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561a6f0c8f00 0x561a6f0c3fd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6ef1e6b0 0x561a6f0c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bee64fb80 1 --2- 172.21.15.82:0/3104772573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a6f0c8a90 0x561a6f0c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 >> 172.21.15.82:0/3104772573 conn(0x561a6f0b83d0 msgr2=0x561a6f0c61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.917+0000 7f2bbb7fe700 0 client.25510 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.918+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 shutdown_connections 2023-07-14T19:33:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.918+0000 7f2bee64fb80 1 -- 172.21.15.82:0/3104772573 wait complete. 2023-07-14T19:33:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.918+0000 7f2bee64fb80 20 client.25510 trim_cache size 0 max 16384 2023-07-14T19:33:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:33:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:33:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:33:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:33:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.947+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2bc40a14d0 msgr2=0x7f2bc40a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.947+0000 7f2be24b7700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2bc40a14d0 0x7f2bc40a3980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4001f30 tx=0x7f2bd4001610 comp rx=0 tx=0).stop 2023-07-14T19:33:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.947+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 msgr2=0x7f2bdc1a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.947+0000 7f2be24b7700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc1a5210 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005f50 tx=0x7f2bcc003cc0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.950+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 shutdown_connections 2023-07-14T19:33:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.950+0000 7f2be24b7700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2bc40a14d0 0x7f2bc40a3980 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.950+0000 7f2be24b7700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc10e3d0 0x7f2bdc1a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.950+0000 7f2be24b7700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bdc06ca40 0x7f2bdc1a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.950+0000 7f2be24b7700 1 --2- 172.21.15.82:0/3012104398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bdc06d4d0 0x7f2bdc1a5210 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.950+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 >> 172.21.15.82:0/3012104398 conn(0x7f2bdc06bea0 msgr2=0x7f2bdc070910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.954+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 shutdown_connections 2023-07-14T19:33:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:11.955+0000 7f2be24b7700 1 -- 172.21.15.82:0/3012104398 wait complete. 2023-07-14T19:33:12.204 INFO:tasks.workunit.client.0.smithi082.stdout: 294 1914588 517 1339730 328 716945 487 2023-07-14T19:33:13.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:12 smithi133 ceph-mon[128894]: pgmap v143: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 582 op/s 2023-07-14T19:33:13.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:12 smithi082 ceph-mon[127461]: pgmap v143: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 582 op/s 2023-07-14T19:33:13.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:12 smithi161 ceph-mon[129184]: pgmap v143: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 582 op/s 2023-07-14T19:33:15.183 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:14 smithi133 ceph-mon[128894]: pgmap v144: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 532 op/s 2023-07-14T19:33:15.225 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:14 smithi082 ceph-mon[127461]: pgmap v144: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 532 op/s 2023-07-14T19:33:15.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:14 smithi161 ceph-mon[129184]: pgmap v144: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 532 op/s 2023-07-14T19:33:17.217 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:16 smithi082 ceph-mon[127461]: pgmap v145: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 863 op/s 2023-07-14T19:33:17.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:16 smithi161 ceph-mon[129184]: pgmap v145: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 863 op/s 2023-07-14T19:33:17.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:16 smithi133 ceph-mon[128894]: pgmap v145: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 11 MiB/s wr, 863 op/s 2023-07-14T19:33:19.215 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:18 smithi082 ceph-mon[127461]: pgmap v146: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 826 op/s 2023-07-14T19:33:19.241 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:18 smithi161 ceph-mon[129184]: pgmap v146: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 826 op/s 2023-07-14T19:33:19.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:18 smithi133 ceph-mon[128894]: pgmap v146: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 11 MiB/s wr, 826 op/s 2023-07-14T19:33:21.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:21 smithi133 ceph-mon[128894]: pgmap v147: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 879 op/s 2023-07-14T19:33:21.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:21 smithi082 ceph-mon[127461]: pgmap v147: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 879 op/s 2023-07-14T19:33:21.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:21 smithi161 ceph-mon[129184]: pgmap v147: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 11 MiB/s wr, 879 op/s 2023-07-14T19:33:22.204 INFO:tasks.workunit.client.0.smithi082.stdout: 301 2665808 644 1875113 420 1002284 1265 2023-07-14T19:33:23.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:23 smithi133 ceph-mon[128894]: pgmap v148: 97 pgs: 97 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.9 MiB/s wr, 753 op/s 2023-07-14T19:33:23.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:23 smithi082 ceph-mon[127461]: pgmap v148: 97 pgs: 97 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.9 MiB/s wr, 753 op/s 2023-07-14T19:33:23.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:23 smithi161 ceph-mon[129184]: pgmap v148: 97 pgs: 97 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.9 MiB/s wr, 753 op/s 2023-07-14T19:33:25.434 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:25 smithi133 ceph-mon[128894]: pgmap v149: 97 pgs: 97 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 472 op/s 2023-07-14T19:33:25.472 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:25 smithi082 ceph-mon[127461]: pgmap v149: 97 pgs: 97 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 472 op/s 2023-07-14T19:33:25.491 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:25 smithi161 ceph-mon[129184]: pgmap v149: 97 pgs: 97 active+clean; 1.2 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.9 MiB/s wr, 472 op/s 2023-07-14T19:33:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.149+0000 7fe4f7678700 1 -- 172.21.15.82:0/2766057503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0107420 msgr2=0x7fe4f010e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.149+0000 7fe4f7678700 1 --2- 172.21.15.82:0/2766057503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0107420 0x7fe4f010e480 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0099c0 tx=0x7fe4ec009cd0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 -- 172.21.15.82:0/2766057503 shutdown_connections 2023-07-14T19:33:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 --2- 172.21.15.82:0/2766057503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f011b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 --2- 172.21.15.82:0/2766057503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0107420 0x7fe4f010e480 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 --2- 172.21.15.82:0/2766057503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f01077f0 0x7fe4f0107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 -- 172.21.15.82:0/2766057503 >> 172.21.15.82:0/2766057503 conn(0x7fe4f006be90 msgr2=0x7fe4f0071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 -- 172.21.15.82:0/2766057503 shutdown_connections 2023-07-14T19:33:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.155+0000 7fe4f7678700 1 -- 172.21.15.82:0/2766057503 wait complete. 2023-07-14T19:33:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.156+0000 7fe4f7678700 1 Processor -- start 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.157+0000 7fe4f7678700 1 -- start start 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.158+0000 7fe4f7678700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f0107420 0x7fe4f010f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.158+0000 7fe4f7678700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f01077f0 0x7fe4f010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.158+0000 7fe4f7678700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f01b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.158+0000 7fe4f7678700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01b9710 con 0x7fe4f01077f0 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.158+0000 7fe4f7678700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01b9880 con 0x7fe4f0107420 2023-07-14T19:33:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.158+0000 7fe4f7678700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01b9a40 con 0x7fe4f0119370 2023-07-14T19:33:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.165+0000 7fe4f5c15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f01b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.165+0000 7fe4f5414700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f0107420 0x7fe4f010f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.165+0000 7fe4f5c15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f01b91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53324/0 (socket says 172.21.15.82:53324) 2023-07-14T19:33:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.165+0000 7fe4f5c15700 1 -- 172.21.15.82:0/447215342 learned_addr learned my addr 172.21.15.82:0/447215342 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.169+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f01077f0 0x7fe4f010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.175+0000 7fe4f5c15700 1 -- 172.21.15.82:0/447215342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f0107420 msgr2=0x7fe4f010f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.175+0000 7fe4f5c15700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f0107420 0x7fe4f010f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.175+0000 7fe4f5c15700 1 -- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f01077f0 msgr2=0x7fe4f010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.175+0000 7fe4f5c15700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f01077f0 0x7fe4f010fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.175+0000 7fe4f5c15700 1 -- 172.21.15.82:0/447215342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0095f0 con 0x7fe4f0119370 2023-07-14T19:33:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.181+0000 7fe4f5c15700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f01b91a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b9e0 tx=0x7fe4e800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.222+0000 7fe4e67fc700 1 -- 172.21.15.82:0/447215342 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800d8e0 con 0x7fe4f0119370 2023-07-14T19:33:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.222+0000 7fe4e67fc700 1 -- 172.21.15.82:0/447215342 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4e80097d0 con 0x7fe4f0119370 2023-07-14T19:33:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.222+0000 7fe4e67fc700 1 -- 172.21.15.82:0/447215342 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80143d0 con 0x7fe4f0119370 2023-07-14T19:33:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.222+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01b9d20 con 0x7fe4f0119370 2023-07-14T19:33:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.222+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01ba180 con 0x7fe4f0119370 2023-07-14T19:33:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.229+0000 7fe503810b80 1 Processor -- start 2023-07-14T19:33:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 -- start start 2023-07-14T19:33:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587599cd6b0 0x558759b77550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 0x558759b71f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759a76c50 con 0x558759b77a90 2023-07-14T19:33:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759ac99e0 con 0x558759b77f00 2023-07-14T19:33:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.231+0000 7fe503810b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759ac2e70 con 0x5587599cd6b0 2023-07-14T19:33:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 0x558759b71f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 0x558759b71f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40414/0 (socket says 172.21.15.82:40414) 2023-07-14T19:33:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 -- 172.21.15.82:0/2121086612 learned_addr learned my addr 172.21.15.82:0/2121086612 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587599cd6b0 0x558759b77550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 -- 172.21.15.82:0/2121086612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587599cd6b0 msgr2=0x558759b77550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587599cd6b0 0x558759b77550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 -- 172.21.15.82:0/2121086612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 msgr2=0x558759b719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.232+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.233+0000 7fe4f4c13700 1 -- 172.21.15.82:0/2121086612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587598f0d80 con 0x558759b77f00 2023-07-14T19:33:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.233+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 0x558759b71f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00149e0 tx=0x7fe4e000c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0ebbd226606760 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.248+0000 7fe4e67fc700 1 -- 172.21.15.82:0/447215342 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe4e8014530 con 0x7fe4f0119370 2023-07-14T19:33:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.249+0000 7fe4e67fc700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe4dc0a14d0 0x7fe4dc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.249+0000 7fe4e67fc700 1 -- 172.21.15.82:0/447215342 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fe4e8011070 con 0x7fe4f0119370 2023-07-14T19:33:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.259+0000 7fe4f5414700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe4dc0a14d0 0x7fe4dc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.282+0000 7fe4db7fe700 1 -- 172.21.15.82:0/2121086612 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00142d0 con 0x558759b77f00 2023-07-14T19:33:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.282+0000 7fe4db7fe700 1 -- 172.21.15.82:0/2121086612 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4e001a070 con 0x558759b77f00 2023-07-14T19:33:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.282+0000 7fe4db7fe700 1 -- 172.21.15.82:0/2121086612 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00126b0 con 0x558759b77f00 2023-07-14T19:33:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 -- 172.21.15.82:0/2121086612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 msgr2=0x558759b71f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 0x558759b71f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00149e0 tx=0x7fe4e000c4f0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 -- 172.21.15.82:0/2121086612 shutdown_connections 2023-07-14T19:33:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77f00 0x558759b71f80 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00149e0 tx=0x7fe4e000c4f0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5587599cd6b0 0x558759b77550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 --2- 172.21.15.82:0/2121086612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.290+0000 7fe503810b80 1 -- 172.21.15.82:0/2121086612 >> 172.21.15.82:0/2121086612 conn(0x558759b673d0 msgr2=0x558759b6c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.294+0000 7fe4f5414700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe4dc0a14d0 0x7fe4dc0a3980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec005c70 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.294+0000 7fe503810b80 1 -- 172.21.15.82:0/2121086612 shutdown_connections 2023-07-14T19:33:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.294+0000 7fe503810b80 1 -- 172.21.15.82:0/2121086612 wait complete. 2023-07-14T19:33:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.295+0000 7fe503810b80 1 -- start start 2023-07-14T19:33:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.295+0000 7fe503810b80 1 Processor -- start 2023-07-14T19:33:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587599cd6b0 0x558759b7b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b87480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759a62a60 con 0x5587599cd6b0 2023-07-14T19:33:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759a62760 con 0x558759b77a90 2023-07-14T19:33:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587599468c0 con 0x558759b7bd50 2023-07-14T19:33:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.296+0000 7fe503810b80 10 client.0 _ll_register_callbacks cb 0x558759a60700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:33:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.301+0000 7fe4f4c13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.301+0000 7fe4f5c15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b87480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.301+0000 7fe4f4c13700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b7b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40422/0 (socket says 172.21.15.82:40422) 2023-07-14T19:33:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.301+0000 7fe4f4c13700 1 -- 172.21.15.82:0/709098037 learned_addr learned my addr 172.21.15.82:0/709098037 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.301+0000 7fe4f5414700 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587599cd6b0 0x558759b7b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.309+0000 7fe4f5414700 1 -- 172.21.15.82:0/709098037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 msgr2=0x558759b87480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.309+0000 7fe4f5414700 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b87480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.309+0000 7fe4f5414700 1 -- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 msgr2=0x558759b7b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.309+0000 7fe4f5414700 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b7b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.309+0000 7fe4f5414700 1 -- 172.21.15.82:0/709098037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e00146c0 con 0x5587599cd6b0 2023-07-14T19:33:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.319+0000 7fe4f5414700 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587599cd6b0 0x558759b7b2d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098e0 tx=0x7fe4ec005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918e6d1675006cda server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.384+0000 7fe4d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:33:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.384+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/709098037 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec02b070 con 0x5587599cd6b0 2023-07-14T19:33:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.384+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/709098037 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fe4ec003e20 con 0x5587599cd6b0 2023-07-14T19:33:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.385+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558759a1c490 con 0x5587599cd6b0 2023-07-14T19:33:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.392+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/709098037 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ec003fe0 con 0x5587599cd6b0 2023-07-14T19:33:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.392+0000 7fe503810b80 10 client.15987 fetch_fsmap learned FSMap version 36 2023-07-14T19:33:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.392+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5587598d9c00 con 0x5587599cd6b0 2023-07-14T19:33:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.397+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/709098037 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558759a1c490 con 0x5587599cd6b0 2023-07-14T19:33:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.397+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/709098037 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe4ec037070 con 0x5587599cd6b0 2023-07-14T19:33:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.398+0000 7fe503810b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:33:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.398+0000 7fe503810b80 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:33:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.398+0000 7fe503810b80 20 client.15987 populate_metadata read hostname 'smithi082' 2023-07-14T19:33:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.398+0000 7fe503810b80 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759b8b8e0 0x558759b8dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.399+0000 7fe503810b80 4 client.15987 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:33:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.399+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558759b40d40 con 0x558759b8b8e0 2023-07-14T19:33:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.399+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759b8b8e0 0x558759b8dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.400+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759b8b8e0 0x558759b8dcf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6926a35daac5c5a1 server_cookie=b0f63db628c4bb51 in_seq=0 out_seq=0 2023-07-14T19:33:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.410+0000 7fe4d8ff9700 10 client.15987 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.410+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/709098037 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558759b40d40 con 0x558759b8b8e0 2023-07-14T19:33:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.410+0000 7fe4d8ff9700 10 client.15987 handle_command_reply: tid=0 2023-07-14T19:33:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.410+0000 7fe503810b80 1 client.15987 shutdown 2023-07-14T19:33:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587599cd6b0 msgr2=0x558759b7b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587599cd6b0 0x558759b7b2d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098e0 tx=0x7fe4ec005c00 comp rx=0 tx=0).stop 2023-07-14T19:33:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 shutdown_connections 2023-07-14T19:33:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759b8b8e0 0x558759b8dcf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b87480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587599cd6b0 0x558759b7b2d0 secure :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0098e0 tx=0x7fe4ec005c00 comp rx=0 tx=0).stop 2023-07-14T19:33:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 --2- 172.21.15.82:0/709098037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 >> 172.21.15.82:0/709098037 conn(0x558759b673d0 msgr2=0x558759b6bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.411+0000 7fe4d8ff9700 0 client.15987 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.414+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 shutdown_connections 2023-07-14T19:33:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.433+0000 7fe503810b80 1 -- 172.21.15.82:0/709098037 wait complete. 2023-07-14T19:33:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.433+0000 7fe503810b80 20 client.15987 trim_cache size 0 max 16384 2023-07-14T19:33:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.516+0000 7fe503810b80 1 Processor -- start 2023-07-14T19:33:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.516+0000 7fe503810b80 1 -- start start 2023-07-14T19:33:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.517+0000 7fe503810b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.517+0000 7fe503810b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.517+0000 7fe503810b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 0x558759b6c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.517+0000 7fe503810b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759b7c760 con 0x558759b76c60 2023-07-14T19:33:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.517+0000 7fe503810b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759b92ab0 con 0x558759b77a90 2023-07-14T19:33:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.517+0000 7fe503810b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759b6d320 con 0x558759b7bd50 2023-07-14T19:33:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.518+0000 7fe4f5c15700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 0x558759b6c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.518+0000 7fe4f5c15700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 0x558759b6c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44080/0 (socket says 172.21.15.82:44080) 2023-07-14T19:33:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.518+0000 7fe4f5c15700 1 -- 172.21.15.82:0/2966041627 learned_addr learned my addr 172.21.15.82:0/2966041627 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.519+0000 7fe4f5414700 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.536+0000 7fe4f5c15700 1 -- 172.21.15.82:0/2966041627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 msgr2=0x558759b871b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T19:33:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.536+0000 7fe4f5c15700 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b871b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.536+0000 7fe4f5c15700 1 -- 172.21.15.82:0/2966041627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 msgr2=0x558759b82400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.536+0000 7fe4f5c15700 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b82400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.536+0000 7fe4f5c15700 1 -- 172.21.15.82:0/2966041627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587599171f0 con 0x558759b76c60 2023-07-14T19:33:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.540+0000 7fe4f5c15700 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 0x558759b6c3b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b9b0 tx=0x7fe4e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a8ab711e80e25 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.561+0000 7fe4dbfff700 1 -- 172.21.15.82:0/2966041627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e809aa30 con 0x558759b76c60 2023-07-14T19:33:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.561+0000 7fe4dbfff700 1 -- 172.21.15.82:0/2966041627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4e809e600 con 0x558759b76c60 2023-07-14T19:33:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.561+0000 7fe4dbfff700 1 -- 172.21.15.82:0/2966041627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80a27f0 con 0x558759b76c60 2023-07-14T19:33:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.562+0000 7fe503810b80 1 -- 172.21.15.82:0/2966041627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 msgr2=0x558759b6c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.562+0000 7fe503810b80 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 0x558759b6c3b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b9b0 tx=0x7fe4e8004ad0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.563+0000 7fe503810b80 1 -- 172.21.15.82:0/2966041627 shutdown_connections 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.563+0000 7fe503810b80 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b76c60 0x558759b6c3b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.563+0000 7fe503810b80 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b77a90 0x558759b82400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.563+0000 7fe503810b80 1 --2- 172.21.15.82:0/2966041627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b7bd50 0x558759b871b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.563+0000 7fe503810b80 1 -- 172.21.15.82:0/2966041627 >> 172.21.15.82:0/2966041627 conn(0x558759b673d0 msgr2=0x558759b86aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.564+0000 7fe503810b80 1 -- 172.21.15.82:0/2966041627 shutdown_connections 2023-07-14T19:33:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.564+0000 7fe503810b80 1 -- 172.21.15.82:0/2966041627 wait complete. 2023-07-14T19:33:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.564+0000 7fe503810b80 1 -- start start 2023-07-14T19:33:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.564+0000 7fe503810b80 1 Processor -- start 2023-07-14T19:33:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b76c60 0x558759b81c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b82180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b7bd50 0x558759b803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759b80a40 con 0x558759b77a90 2023-07-14T19:33:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759b80bb0 con 0x558759b7bd50 2023-07-14T19:33:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558759b80d70 con 0x558759b76c60 2023-07-14T19:33:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.566+0000 7fe503810b80 10 client.0 _ll_register_callbacks cb 0x7fe4ec005660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:33:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.571+0000 7fe4f5414700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b76c60 0x558759b81c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.571+0000 7fe4f5c15700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b7bd50 0x558759b803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.571+0000 7fe4f4c13700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b82180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.571+0000 7fe4f4c13700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b82180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44088/0 (socket says 172.21.15.82:44088) 2023-07-14T19:33:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.571+0000 7fe4f4c13700 1 -- 172.21.15.82:0/4131711964 learned_addr learned my addr 172.21.15.82:0/4131711964 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.592+0000 7fe4f4c13700 1 -- 172.21.15.82:0/4131711964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b76c60 msgr2=0x558759b81c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.592+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b76c60 0x558759b81c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.592+0000 7fe4f4c13700 1 -- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b7bd50 msgr2=0x558759b803f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.592+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b7bd50 0x558759b803f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.592+0000 7fe4f4c13700 1 -- 172.21.15.82:0/4131711964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01ba180 con 0x558759b77a90 2023-07-14T19:33:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.598+0000 7fe4f4c13700 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b82180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00149b0 tx=0x7fe4e000c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3895784557c111ee server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.616+0000 7fe4d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:33:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.616+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/4131711964 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e002b070 con 0x558759b77a90 2023-07-14T19:33:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.617+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/4131711964 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fe4e0011ce0 con 0x558759b77a90 2023-07-14T19:33:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.617+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558759b882d0 con 0x558759b77a90 2023-07-14T19:33:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.627+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/4131711964 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e0024040 con 0x558759b77a90 2023-07-14T19:33:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.627+0000 7fe503810b80 10 client.15993 fetch_fsmap learned FSMap version 36 2023-07-14T19:33:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.627+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5587598ef250 con 0x558759b77a90 2023-07-14T19:33:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/4131711964 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558759b882d0 con 0x558759b77a90 2023-07-14T19:33:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/4131711964 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe4e002e020 con 0x558759b77a90 2023-07-14T19:33:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe503810b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:33:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe503810b80 10 client.15993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:33:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe503810b80 20 client.15993 populate_metadata read hostname 'smithi082' 2023-07-14T19:33:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe503810b80 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759be0590 0x558759be2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe503810b80 4 client.15993 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:33:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.633+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe4e001a070 con 0x558759be0590 2023-07-14T19:33:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.657+0000 7fe4f5414700 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759be0590 0x558759be2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.688+0000 7fe4f5414700 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759be0590 0x558759be2a40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe99a304b6d885bd server_cookie=d7dfe7f2fa6dd32a in_seq=0 out_seq=0 2023-07-14T19:33:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:26.692+0000 7fe4d8ff9700 10 client.15993 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:27.433 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:27 smithi133 ceph-mon[128894]: pgmap v150: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 597 op/s 2023-07-14T19:33:27.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:27 smithi082 ceph-mon[127461]: pgmap v150: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 597 op/s 2023-07-14T19:33:27.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:27 smithi161 ceph-mon[129184]: pgmap v150: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 597 op/s 2023-07-14T19:33:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.010+0000 7fe4d8ff9700 1 -- 172.21.15.82:0/4131711964 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe4e001a070 con 0x558759be0590 2023-07-14T19:33:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.010+0000 7fe4d8ff9700 10 client.15993 handle_command_reply: tid=0 2023-07-14T19:33:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.010+0000 7fe503810b80 1 client.15993 shutdown 2023-07-14T19:33:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.010+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 msgr2=0x558759b82180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.010+0000 7fe503810b80 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b82180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00149b0 tx=0x7fe4e000c5c0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 shutdown_connections 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558759b7bd50 0x558759b803f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558759be0590 0x558759be2a40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558759b76c60 0x558759b81c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 --2- 172.21.15.82:0/4131711964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558759b77a90 0x558759b82180 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 >> 172.21.15.82:0/4131711964 conn(0x558759b673d0 msgr2=0x558759b87330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe4d8ff9700 0 client.15993 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.011+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 shutdown_connections 2023-07-14T19:33:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.012+0000 7fe503810b80 1 -- 172.21.15.82:0/4131711964 wait complete. 2023-07-14T19:33:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.012+0000 7fe503810b80 20 client.15993 trim_cache size 0 max 16384 2023-07-14T19:33:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:33:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:33:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:33:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:33:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe4dc0a14d0 msgr2=0x7fe4dc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe4dc0a14d0 0x7fe4dc0a3980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec005c70 comp rx=0 tx=0).stop 2023-07-14T19:33:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 msgr2=0x7fe4f01b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f01b91a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b9e0 tx=0x7fe4e800bcf0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 shutdown_connections 2023-07-14T19:33:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe4dc0a14d0 0x7fe4dc0a3980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4f0119370 0x7fe4f01b91a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f0107420 0x7fe4f010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 --2- 172.21.15.82:0/447215342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f01077f0 0x7fe4f010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.141+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 >> 172.21.15.82:0/447215342 conn(0x7fe4f006be90 msgr2=0x7fe4f0071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.142+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 shutdown_connections 2023-07-14T19:33:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:28.142+0000 7fe4f7678700 1 -- 172.21.15.82:0/447215342 wait complete. 2023-07-14T19:33:29.492 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:29 smithi161 ceph-mon[129184]: pgmap v151: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2023-07-14T19:33:29.687 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:29 smithi133 ceph-mon[128894]: pgmap v151: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2023-07-14T19:33:29.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:29 smithi082 ceph-mon[127461]: pgmap v151: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 283 op/s 2023-07-14T19:33:31.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:31 smithi133 ceph-mon[128894]: pgmap v152: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2023-07-14T19:33:31.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:31 smithi082 ceph-mon[127461]: pgmap v152: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2023-07-14T19:33:31.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:31 smithi161 ceph-mon[129184]: pgmap v152: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 465 op/s 2023-07-14T19:33:32.213 INFO:tasks.workunit.client.0.smithi082.stdout: 308 2366044 468 1652882 382 906069 913 2023-07-14T19:33:33.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:33 smithi133 ceph-mon[128894]: pgmap v153: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.6 MiB/s wr, 802 op/s 2023-07-14T19:33:33.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:33 smithi082 ceph-mon[127461]: pgmap v153: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.6 MiB/s wr, 802 op/s 2023-07-14T19:33:33.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:33 smithi161 ceph-mon[129184]: pgmap v153: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.6 MiB/s wr, 802 op/s 2023-07-14T19:33:35.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:35 smithi133 ceph-mon[128894]: pgmap v154: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 749 op/s 2023-07-14T19:33:35.715 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:35 smithi082 ceph-mon[127461]: pgmap v154: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 749 op/s 2023-07-14T19:33:35.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:35 smithi161 ceph-mon[129184]: pgmap v154: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.0 MiB/s wr, 749 op/s 2023-07-14T19:33:37.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:37 smithi133 ceph-mon[128894]: pgmap v155: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.5 MiB/s wr, 872 op/s 2023-07-14T19:33:37.716 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:37 smithi082 ceph-mon[127461]: pgmap v155: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.5 MiB/s wr, 872 op/s 2023-07-14T19:33:37.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:37 smithi161 ceph-mon[129184]: pgmap v155: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.5 MiB/s wr, 872 op/s 2023-07-14T19:33:39.684 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:39 smithi133 ceph-mon[128894]: pgmap v156: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.5 MiB/s wr, 767 op/s 2023-07-14T19:33:39.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:39 smithi082 ceph-mon[127461]: pgmap v156: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.5 MiB/s wr, 767 op/s 2023-07-14T19:33:39.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:39 smithi161 ceph-mon[129184]: pgmap v156: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.5 MiB/s wr, 767 op/s 2023-07-14T19:33:41.685 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:41 smithi133 ceph-mon[128894]: pgmap v157: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 760 op/s 2023-07-14T19:33:41.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:41 smithi082 ceph-mon[127461]: pgmap v157: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 760 op/s 2023-07-14T19:33:41.742 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:41 smithi161 ceph-mon[129184]: pgmap v157: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.7 MiB/s wr, 760 op/s 2023-07-14T19:33:42.217 INFO:tasks.workunit.client.0.smithi082.stdout: 318 2738793 496 1913944 432 1036305 820 2023-07-14T19:33:42.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:42 smithi161 ceph-mon[129184]: pgmap v158: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 569 op/s 2023-07-14T19:33:42.934 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:42 smithi133 ceph-mon[128894]: pgmap v158: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 569 op/s 2023-07-14T19:33:42.974 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:42 smithi082 ceph-mon[127461]: pgmap v158: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.0 MiB/s wr, 569 op/s 2023-07-14T19:33:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.628+0000 7fa088510700 1 -- 172.21.15.82:0/2512868299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 msgr2=0x7fa0800788a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:43.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.628+0000 7fa088510700 1 --2- 172.21.15.82:0/2512868299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa0800788a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa078009fc0 tx=0x7fa07800b330 comp rx=0 tx=0).stop 2023-07-14T19:33:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 -- 172.21.15.82:0/2512868299 shutdown_connections 2023-07-14T19:33:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 --2- 172.21.15.82:0/2512868299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa0800788a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 --2- 172.21.15.82:0/2512868299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08010c2f0 0x7fa08010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 --2- 172.21.15.82:0/2512868299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08006cbb0 0x7fa08006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 -- 172.21.15.82:0/2512868299 >> 172.21.15.82:0/2512868299 conn(0x7fa08006bea0 msgr2=0x7fa080071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:43.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 -- 172.21.15.82:0/2512868299 shutdown_connections 2023-07-14T19:33:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.634+0000 7fa088510700 1 -- 172.21.15.82:0/2512868299 wait complete. 2023-07-14T19:33:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.635+0000 7fa088510700 1 Processor -- start 2023-07-14T19:33:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.636+0000 7fa088510700 1 -- start start 2023-07-14T19:33:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa088510700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08006cbb0 0x7fa080081e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa088510700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa08007bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa088510700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 0x7fa08007c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa088510700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08007c6b0 con 0x7fa08010c2f0 2023-07-14T19:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa088510700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08007c820 con 0x7fa08006cbb0 2023-07-14T19:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa088510700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08007c9e0 con 0x7fa08006d540 2023-07-14T19:33:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.637+0000 7fa0862ac700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08006cbb0 0x7fa080081e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.642+0000 7fa086aad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 0x7fa08007c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.642+0000 7fa086aad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 0x7fa08007c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44104/0 (socket says 172.21.15.82:44104) 2023-07-14T19:33:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.642+0000 7fa086aad700 1 -- 172.21.15.82:0/356370966 learned_addr learned my addr 172.21.15.82:0/356370966 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.645+0000 7fa085aab700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa08007bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa086aad700 1 -- 172.21.15.82:0/356370966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 msgr2=0x7fa08007bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa086aad700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa08007bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa086aad700 1 -- 172.21.15.82:0/356370966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08006cbb0 msgr2=0x7fa080081e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa086aad700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08006cbb0 0x7fa080081e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa086aad700 1 -- 172.21.15.82:0/356370966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa078009c40 con 0x7fa08010c2f0 2023-07-14T19:33:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa085aab700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa08007bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:33:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.656+0000 7fa0862ac700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08006cbb0 0x7fa080081e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:33:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.666+0000 7fa086aad700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 0x7fa08007c140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa078009040 tx=0x7fa078004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.692+0000 7fa0777fe700 1 -- 172.21.15.82:0/356370966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07800e040 con 0x7fa08010c2f0 2023-07-14T19:33:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.693+0000 7fa0777fe700 1 -- 172.21.15.82:0/356370966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07800bbf0 con 0x7fa08010c2f0 2023-07-14T19:33:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.693+0000 7fa0777fe700 1 -- 172.21.15.82:0/356370966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07801b6c0 con 0x7fa08010c2f0 2023-07-14T19:33:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.693+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0801cafe0 con 0x7fa08010c2f0 2023-07-14T19:33:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.693+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0801cb3b0 con 0x7fa08010c2f0 2023-07-14T19:33:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.700+0000 7fa0946a8b80 1 Processor -- start 2023-07-14T19:33:43.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.703+0000 7fa0946a8b80 1 -- start start 2023-07-14T19:33:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b090550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b08a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b08af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37af8fc50 con 0x55e37b090a90 2023-07-14T19:33:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37afe29e0 con 0x55e37aee66b0 2023-07-14T19:33:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37afdbe70 con 0x55e37b090f00 2023-07-14T19:33:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b090550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b090550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40454/0 (socket says 172.21.15.82:40454) 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 -- 172.21.15.82:0/253322186 learned_addr learned my addr 172.21.15.82:0/253322186 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b08af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 -- 172.21.15.82:0/253322186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 msgr2=0x55e37b08af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b08af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 -- 172.21.15.82:0/253322186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 msgr2=0x55e37b08a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.704+0000 7fa085aab700 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b08a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.705+0000 7fa085aab700 1 -- 172.21.15.82:0/253322186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37ae09d80 con 0x55e37aee66b0 2023-07-14T19:33:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.705+0000 7fa0777fe700 1 -- 172.21.15.82:0/356370966 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa07801b820 con 0x7fa08010c2f0 2023-07-14T19:33:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.705+0000 7fa085aab700 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b090550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa070012bd0 tx=0x7fa07000bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eedfbc458a107bd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.706+0000 7fa0777fe700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa06c0a14d0 0x7fa06c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.706+0000 7fa0777fe700 1 -- 172.21.15.82:0/356370966 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fa0780cede0 con 0x7fa08010c2f0 2023-07-14T19:33:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.708+0000 7fa0862ac700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa06c0a14d0 0x7fa06c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.734+0000 7fa0862ac700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa06c0a14d0 0x7fa06c0a3980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005fd0 tx=0x7fa07c009770 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.796+0000 7fa074ff9700 1 -- 172.21.15.82:0/253322186 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070004020 con 0x55e37aee66b0 2023-07-14T19:33:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.796+0000 7fa074ff9700 1 -- 172.21.15.82:0/253322186 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa070004830 con 0x55e37aee66b0 2023-07-14T19:33:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.796+0000 7fa074ff9700 1 -- 172.21.15.82:0/253322186 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070005260 con 0x55e37aee66b0 2023-07-14T19:33:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.797+0000 7fa0946a8b80 1 -- 172.21.15.82:0/253322186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 msgr2=0x55e37b090550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.797+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b090550 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa070012bd0 tx=0x7fa07000bbb0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.819+0000 7fa0946a8b80 1 -- 172.21.15.82:0/253322186 shutdown_connections 2023-07-14T19:33:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.819+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b08af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.819+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b090550 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.819+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/253322186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b08a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.819+0000 7fa0946a8b80 1 -- 172.21.15.82:0/253322186 >> 172.21.15.82:0/253322186 conn(0x55e37b0803d0 msgr2=0x55e37b085530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.820+0000 7fa0946a8b80 1 -- 172.21.15.82:0/253322186 shutdown_connections 2023-07-14T19:33:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.820+0000 7fa0946a8b80 1 -- 172.21.15.82:0/253322186 wait complete. 2023-07-14T19:33:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.820+0000 7fa0946a8b80 1 -- start start 2023-07-14T19:33:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 Processor -- start 2023-07-14T19:33:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b096e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b0940d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b0947c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37af7ba60 con 0x55e37b090a90 2023-07-14T19:33:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37af7b760 con 0x55e37aee66b0 2023-07-14T19:33:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37ae5f8c0 con 0x55e37b090f00 2023-07-14T19:33:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.821+0000 7fa0946a8b80 10 client.0 _ll_register_callbacks cb 0x55e37af79700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:33:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.824+0000 7fa0862ac700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b096e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.824+0000 7fa0862ac700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b096e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40464/0 (socket says 172.21.15.82:40464) 2023-07-14T19:33:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.824+0000 7fa0862ac700 1 -- 172.21.15.82:0/892552001 learned_addr learned my addr 172.21.15.82:0/892552001 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.824+0000 7fa086aad700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b0947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.826+0000 7fa085aab700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b0940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.835+0000 7fa0862ac700 1 -- 172.21.15.82:0/892552001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 msgr2=0x55e37b0947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.835+0000 7fa0862ac700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b0947c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.835+0000 7fa0862ac700 1 -- 172.21.15.82:0/892552001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 msgr2=0x55e37b0940d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.835+0000 7fa0862ac700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b0940d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.835+0000 7fa0862ac700 1 -- 172.21.15.82:0/892552001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07001a040 con 0x55e37aee66b0 2023-07-14T19:33:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.844+0000 7fa0862ac700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b096e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa07c012870 tx=0x7fa07c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a609b29cf541752 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.972+0000 7fa069ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:33:43.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.972+0000 7fa069ffb700 1 -- 172.21.15.82:0/892552001 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c01f630 con 0x55e37aee66b0 2023-07-14T19:33:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.972+0000 7fa069ffb700 1 -- 172.21.15.82:0/892552001 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fa07c018070 con 0x55e37aee66b0 2023-07-14T19:33:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.979+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e37af35490 con 0x55e37aee66b0 2023-07-14T19:33:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.989+0000 7fa069ffb700 1 -- 172.21.15.82:0/892552001 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa07c01f790 con 0x55e37aee66b0 2023-07-14T19:33:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.989+0000 7fa0946a8b80 10 client.25691 fetch_fsmap learned FSMap version 36 2023-07-14T19:33:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.989+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e37adf2c00 con 0x55e37aee66b0 2023-07-14T19:33:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.998+0000 7fa069ffb700 1 -- 172.21.15.82:0/892552001 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e37af35490 con 0x55e37aee66b0 2023-07-14T19:33:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.998+0000 7fa069ffb700 1 -- 172.21.15.82:0/892552001 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa07c031c20 con 0x55e37aee66b0 2023-07-14T19:33:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.998+0000 7fa0946a8b80 10 client.25691 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:33:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.998+0000 7fa0946a8b80 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:33:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.999+0000 7fa0946a8b80 20 client.25691 populate_metadata read hostname 'smithi082' 2023-07-14T19:33:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.999+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b0a4470 0x55e37b0a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.999+0000 7fa0946a8b80 4 client.25691 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:33:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:43.999+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e37b059d40 con 0x55e37b0a4470 2023-07-14T19:33:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.010+0000 7fa085aab700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b0a4470 0x55e37b0a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.028+0000 7fa085aab700 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b0a4470 0x55e37b0a6880 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de811e0563b2f2d0 server_cookie=799727c3ff0c8f9f in_seq=0 out_seq=0 2023-07-14T19:33:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.039+0000 7fa069ffb700 10 client.25691 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.039+0000 7fa069ffb700 1 -- 172.21.15.82:0/892552001 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55e37b059d40 con 0x55e37b0a4470 2023-07-14T19:33:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.039+0000 7fa069ffb700 10 client.25691 handle_command_reply: tid=0 2023-07-14T19:33:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.039+0000 7fa0946a8b80 1 client.25691 shutdown 2023-07-14T19:33:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 msgr2=0x55e37b096e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b096e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa07c012870 tx=0x7fa07c009310 comp rx=0 tx=0).stop 2023-07-14T19:33:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 shutdown_connections 2023-07-14T19:33:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b0a4470 0x55e37b0a6880 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37b090f00 0x55e37b0947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37aee66b0 0x55e37b096e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/892552001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b0940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 >> 172.21.15.82:0/892552001 conn(0x55e37b0803d0 msgr2=0x55e37b084c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa069ffb700 0 client.25691 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 shutdown_connections 2023-07-14T19:33:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.040+0000 7fa0946a8b80 1 -- 172.21.15.82:0/892552001 wait complete. 2023-07-14T19:33:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.041+0000 7fa0946a8b80 20 client.25691 trim_cache size 0 max 16384 2023-07-14T19:33:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.283+0000 7fa0946a8b80 1 Processor -- start 2023-07-14T19:33:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 -- start start 2023-07-14T19:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b0803d0 0x55e37b08f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b0853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b090a90 0x55e37b091a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37b094e30 con 0x55e37b0803d0 2023-07-14T19:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37b086300 con 0x55e37b090a90 2023-07-14T19:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.301+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37b086770 con 0x55e37aee66b0 2023-07-14T19:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.302+0000 7fa085aab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b0853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.302+0000 7fa085aab700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b0853b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53388/0 (socket says 172.21.15.82:53388) 2023-07-14T19:33:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.302+0000 7fa085aab700 1 -- 172.21.15.82:0/3298151132 learned_addr learned my addr 172.21.15.82:0/3298151132 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.302+0000 7fa086aad700 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b090a90 0x55e37b091a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.302+0000 7fa0862ac700 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b0803d0 0x55e37b08f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.310+0000 7fa086aad700 1 -- 172.21.15.82:0/3298151132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 msgr2=0x55e37b0853b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.310+0000 7fa086aad700 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b0853b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.310+0000 7fa086aad700 1 -- 172.21.15.82:0/3298151132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b0803d0 msgr2=0x55e37b08f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.310+0000 7fa086aad700 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b0803d0 0x55e37b08f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.310+0000 7fa086aad700 1 -- 172.21.15.82:0/3298151132 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37ae301f0 con 0x55e37b090a90 2023-07-14T19:33:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.316+0000 7fa086aad700 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b090a90 0x55e37b091a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0780cc560 tx=0x7fa078009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41b2f1b90f6cd477 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:44.329 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.326+0000 7fa0757fa700 1 -- 172.21.15.82:0/3298151132 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07809bed0 con 0x55e37b090a90 2023-07-14T19:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.326+0000 7fa0757fa700 1 -- 172.21.15.82:0/3298151132 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa07809ff00 con 0x55e37b090a90 2023-07-14T19:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.326+0000 7fa0757fa700 1 -- 172.21.15.82:0/3298151132 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07809c390 con 0x55e37b090a90 2023-07-14T19:33:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.327+0000 7fa0946a8b80 1 -- 172.21.15.82:0/3298151132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b090a90 msgr2=0x55e37b091a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.327+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b090a90 0x55e37b091a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0780cc560 tx=0x7fa078009920 comp rx=0 tx=0).stop 2023-07-14T19:33:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.328+0000 7fa0946a8b80 1 -- 172.21.15.82:0/3298151132 shutdown_connections 2023-07-14T19:33:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.328+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b090a90 0x55e37b091a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.328+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b0803d0 0x55e37b08f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.328+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/3298151132 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b0853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.328+0000 7fa0946a8b80 1 -- 172.21.15.82:0/3298151132 >> 172.21.15.82:0/3298151132 conn(0x55e37b09fe90 msgr2=0x55e37b091320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.329+0000 7fa0946a8b80 1 -- 172.21.15.82:0/3298151132 shutdown_connections 2023-07-14T19:33:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.329+0000 7fa0946a8b80 1 -- 172.21.15.82:0/3298151132 wait complete. 2023-07-14T19:33:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.329+0000 7fa0946a8b80 1 -- start start 2023-07-14T19:33:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.329+0000 7fa0946a8b80 1 Processor -- start 2023-07-14T19:33:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.330+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b093a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.331+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 0x55e37b093fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.331+0000 7fa0946a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b092650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.331+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37b092c50 con 0x55e37b090a90 2023-07-14T19:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.331+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37b092dc0 con 0x55e37b0803d0 2023-07-14T19:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.331+0000 7fa0946a8b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37b092f80 con 0x55e37aee66b0 2023-07-14T19:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.331+0000 7fa0946a8b80 10 client.0 _ll_register_callbacks cb 0x7fa07c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:33:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.332+0000 7fa085aab700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 0x55e37b093fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.332+0000 7fa085aab700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 0x55e37b093fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40478/0 (socket says 172.21.15.82:40478) 2023-07-14T19:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.332+0000 7fa085aab700 1 -- 172.21.15.82:0/1450646037 learned_addr learned my addr 172.21.15.82:0/1450646037 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.332+0000 7fa0862ac700 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b093a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.333+0000 7fa085aab700 1 -- 172.21.15.82:0/1450646037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 msgr2=0x55e37b093a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.333+0000 7fa085aab700 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b093a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.333+0000 7fa085aab700 1 -- 172.21.15.82:0/1450646037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 msgr2=0x55e37b092650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.333+0000 7fa085aab700 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b092650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.333+0000 7fa085aab700 1 -- 172.21.15.82:0/1450646037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0780a1040 con 0x55e37b0803d0 2023-07-14T19:33:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.333+0000 7fa085aab700 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 0x55e37b093fa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa070012ba0 tx=0x7fa070012740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126bc5b5f9539d28 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:33:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.348+0000 7fa069ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:33:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.348+0000 7fa069ffb700 1 -- 172.21.15.82:0/1450646037 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070024040 con 0x55e37b0803d0 2023-07-14T19:33:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.348+0000 7fa069ffb700 1 -- 172.21.15.82:0/1450646037 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fa070004d20 con 0x55e37b0803d0 2023-07-14T19:33:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.348+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa070004830 con 0x55e37b0803d0 2023-07-14T19:33:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.362+0000 7fa069ffb700 1 -- 172.21.15.82:0/1450646037 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa070004020 con 0x55e37b0803d0 2023-07-14T19:33:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.362+0000 7fa0946a8b80 10 client.25697 fetch_fsmap learned FSMap version 36 2023-07-14T19:33:44.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.362+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55e37ae08250 con 0x55e37b0803d0 2023-07-14T19:33:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.369+0000 7fa069ffb700 1 -- 172.21.15.82:0/1450646037 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa070004830 con 0x55e37b0803d0 2023-07-14T19:33:44.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.369+0000 7fa069ffb700 1 -- 172.21.15.82:0/1450646037 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa070005000 con 0x55e37b0803d0 2023-07-14T19:33:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.370+0000 7fa0946a8b80 10 client.25697 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:33:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.370+0000 7fa0946a8b80 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:33:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.370+0000 7fa0946a8b80 20 client.25697 populate_metadata read hostname 'smithi082' 2023-07-14T19:33:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.370+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b090f00 0x55e37b0fb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.370+0000 7fa0946a8b80 4 client.25697 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:33:44.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.370+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e37afd4e50 con 0x55e37b090f00 2023-07-14T19:33:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.380+0000 7fa0862ac700 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b090f00 0x55e37b0fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:33:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.404+0000 7fa0862ac700 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b090f00 0x55e37b0fb000 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36619d23abd459f server_cookie=da9f6d5d470fe645 in_seq=0 out_seq=0 2023-07-14T19:33:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:44.404+0000 7fa069ffb700 10 client.25697 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:45.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:44 smithi082 ceph-mon[127461]: pgmap v159: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2023-07-14T19:33:45.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:44 smithi161 ceph-mon[129184]: pgmap v159: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2023-07-14T19:33:45.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:44 smithi133 ceph-mon[128894]: pgmap v159: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2023-07-14T19:33:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.641+0000 7fa069ffb700 1 -- 172.21.15.82:0/1450646037 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e37afd4e50 con 0x55e37b090f00 2023-07-14T19:33:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.641+0000 7fa069ffb700 10 client.25697 handle_command_reply: tid=0 2023-07-14T19:33:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.641+0000 7fa0946a8b80 1 client.25697 shutdown 2023-07-14T19:33:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.642+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 msgr2=0x55e37b093fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:45.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.642+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 0x55e37b093fa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa070012ba0 tx=0x7fa070012740 comp rx=0 tx=0).stop 2023-07-14T19:33:45.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.654+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 shutdown_connections 2023-07-14T19:33:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.654+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e37b090a90 0x55e37b092650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.654+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55e37b090f00 0x55e37b0fb000 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.654+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e37aee66b0 0x55e37b093a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.654+0000 7fa0946a8b80 1 --2- 172.21.15.82:0/1450646037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37b0803d0 0x55e37b093fa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.654+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 >> 172.21.15.82:0/1450646037 conn(0x55e37b09fe90 msgr2=0x55e37b08f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:45.689 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.664+0000 7fa069ffb700 0 client.25697 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:33:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.686+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 shutdown_connections 2023-07-14T19:33:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.686+0000 7fa0946a8b80 1 -- 172.21.15.82:0/1450646037 wait complete. 2023-07-14T19:33:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.687+0000 7fa0946a8b80 20 client.25697 trim_cache size 0 max 16384 2023-07-14T19:33:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:33:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:33:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:33:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:33:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa06c0a14d0 msgr2=0x7fa06c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa06c0a14d0 0x7fa06c0a3980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005fd0 tx=0x7fa07c009770 comp rx=0 tx=0).stop 2023-07-14T19:33:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 msgr2=0x7fa08007c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 0x7fa08007c140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa078009040 tx=0x7fa078004a30 comp rx=0 tx=0).stop 2023-07-14T19:33:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 shutdown_connections 2023-07-14T19:33:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa06c0a14d0 0x7fa06c0a3980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa08010c2f0 0x7fa08007c140 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa08006cbb0 0x7fa080081e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 --2- 172.21.15.82:0/356370966 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa08006d540 0x7fa08007bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.887+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 >> 172.21.15.82:0/356370966 conn(0x7fa08006bea0 msgr2=0x7fa080071170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.888+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 shutdown_connections 2023-07-14T19:33:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:45.888+0000 7fa088510700 1 -- 172.21.15.82:0/356370966 wait complete. 2023-07-14T19:33:47.216 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:46 smithi082 ceph-mon[127461]: pgmap v160: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2023-07-14T19:33:47.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:46 smithi161 ceph-mon[129184]: pgmap v160: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2023-07-14T19:33:47.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:46 smithi133 ceph-mon[128894]: pgmap v160: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2023-07-14T19:33:49.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:49 smithi133 ceph-mon[128894]: pgmap v161: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2023-07-14T19:33:49.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:49 smithi082 ceph-mon[127461]: pgmap v161: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2023-07-14T19:33:49.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:49 smithi161 ceph-mon[129184]: pgmap v161: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2023-07-14T19:33:51.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:51 smithi133 ceph-mon[128894]: pgmap v162: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 364 op/s 2023-07-14T19:33:51.482 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:51 smithi082 ceph-mon[127461]: pgmap v162: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 364 op/s 2023-07-14T19:33:51.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:51 smithi161 ceph-mon[129184]: pgmap v162: 97 pgs: 97 active+clean; 1.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 364 op/s 2023-07-14T19:33:52.224 INFO:tasks.workunit.client.0.smithi082.stdout: 324 2377921 536 1643184 323 909424 589 2023-07-14T19:33:53.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:53 smithi133 ceph-mon[128894]: pgmap v163: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 616 op/s 2023-07-14T19:33:53.466 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:53 smithi082 ceph-mon[127461]: pgmap v163: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 616 op/s 2023-07-14T19:33:53.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:53 smithi161 ceph-mon[129184]: pgmap v163: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 616 op/s 2023-07-14T19:33:55.435 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:55 smithi133 ceph-mon[128894]: pgmap v164: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.8 MiB/s wr, 597 op/s 2023-07-14T19:33:55.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:55 smithi161 ceph-mon[129184]: pgmap v164: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.8 MiB/s wr, 597 op/s 2023-07-14T19:33:55.496 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:55 smithi082 ceph-mon[127461]: pgmap v164: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.8 MiB/s wr, 597 op/s 2023-07-14T19:33:57.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:57 smithi133 ceph-mon[128894]: pgmap v165: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.5 MiB/s wr, 688 op/s 2023-07-14T19:33:57.436 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:57 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:33:57.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:57 smithi082 ceph-mon[127461]: pgmap v165: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.5 MiB/s wr, 688 op/s 2023-07-14T19:33:57.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:57 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:33:57.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:57 smithi161 ceph-mon[129184]: pgmap v165: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.5 MiB/s wr, 688 op/s 2023-07-14T19:33:57.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:57 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:33:59.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:33:59 smithi082 ceph-mon[127461]: pgmap v166: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2023-07-14T19:33:59.494 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:33:59 smithi161 ceph-mon[129184]: pgmap v166: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2023-07-14T19:33:59.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:33:59 smithi133 ceph-mon[128894]: pgmap v166: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 511 op/s 2023-07-14T19:33:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.969+0000 7f10db9a3700 1 -- 172.21.15.82:0/2320926624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d40712b0 msgr2=0x7f10d4071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:33:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.969+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2320926624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d40712b0 0x7f10d4071720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f10c40099c0 tx=0x7f10c4009cd0 comp rx=0 tx=0).stop 2023-07-14T19:33:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 -- 172.21.15.82:0/2320926624 shutdown_connections 2023-07-14T19:33:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2320926624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10d4071c60 0x7f10d411b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2320926624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d410e160 0x7f10d410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2320926624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d40712b0 0x7f10d4071720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:33:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 -- 172.21.15.82:0/2320926624 >> 172.21.15.82:0/2320926624 conn(0x7f10d406bf30 msgr2=0x7f10d406c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:33:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 -- 172.21.15.82:0/2320926624 shutdown_connections 2023-07-14T19:33:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.971+0000 7f10db9a3700 1 -- 172.21.15.82:0/2320926624 wait complete. 2023-07-14T19:33:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.972+0000 7f10db9a3700 1 Processor -- start 2023-07-14T19:33:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.972+0000 7f10db9a3700 1 -- start start 2023-07-14T19:33:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.973+0000 7f10db9a3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10d40712b0 0x7f10d4113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.973+0000 7f10db9a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d4071c60 0x7f10d4114300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.973+0000 7f10db9a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d410e160 0x7f10d41b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:33:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.973+0000 7f10db9a3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41b9560 con 0x7f10d4071c60 2023-07-14T19:33:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.973+0000 7f10db9a3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41b96d0 con 0x7f10d40712b0 2023-07-14T19:33:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:33:59.973+0000 7f10db9a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41b9890 con 0x7f10d410e160 2023-07-14T19:34:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.015+0000 7f10d8f3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d4071c60 0x7f10d4114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.015+0000 7f10d8f3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d4071c60 0x7f10d4114300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44140/0 (socket says 172.21.15.82:44140) 2023-07-14T19:34:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.015+0000 7f10d8f3e700 1 -- 172.21.15.82:0/2591781264 learned_addr learned my addr 172.21.15.82:0/2591781264 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.015+0000 7f10d973f700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10d40712b0 0x7f10d4113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.015+0000 7f10d9f40700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d410e160 0x7f10d41b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10d9f40700 1 -- 172.21.15.82:0/2591781264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10d40712b0 msgr2=0x7f10d4113dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10d9f40700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10d40712b0 0x7f10d4113dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10d9f40700 1 -- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d4071c60 msgr2=0x7f10d4114300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10d9f40700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d4071c60 0x7f10d4114300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10d9f40700 1 -- 172.21.15.82:0/2591781264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40095f0 con 0x7f10d410e160 2023-07-14T19:34:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10d9f40700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d410e160 0x7f10d41b8f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c7a0 tx=0x7f10d000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10ca7fc700 1 -- 172.21.15.82:0/2591781264 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d0004a30 con 0x7f10d410e160 2023-07-14T19:34:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.016+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d41b9b70 con 0x7f10d410e160 2023-07-14T19:34:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.017+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d41ba080 con 0x7f10d410e160 2023-07-14T19:34:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.023+0000 7f10e7b3bb80 1 Processor -- start 2023-07-14T19:34:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.023+0000 7f10ca7fc700 1 -- 172.21.15.82:0/2591781264 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10d00054e0 con 0x7f10d410e160 2023-07-14T19:34:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.023+0000 7f10ca7fc700 1 -- 172.21.15.82:0/2591781264 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d000eb70 con 0x7f10d410e160 2023-07-14T19:34:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.026+0000 7f10ca7fc700 1 -- 172.21.15.82:0/2591781264 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f10d0004b90 con 0x7f10d410e160 2023-07-14T19:34:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.028+0000 7f10ca7fc700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f10c00a1400 0x7f10c00a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.028+0000 7f10ca7fc700 1 -- 172.21.15.82:0/2591781264 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f10d00cc5e0 con 0x7f10d410e160 2023-07-14T19:34:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.033+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f10c00a1400 0x7f10c00a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.038+0000 7f10e7b3bb80 1 -- start start 2023-07-14T19:34:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.038+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee726e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.038+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7268c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.038+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.038+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee716dc50 con 0x558ee7269170 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.038+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee71c09e0 con 0x558ee70c46b0 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.039+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee71b9e70 con 0x558ee726ecc0 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.039+0000 7f10d8f3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7268c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.039+0000 7f10d8f3e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7268c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53412/0 (socket says 172.21.15.82:53412) 2023-07-14T19:34:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.039+0000 7f10d8f3e700 1 -- 172.21.15.82:0/3739772495 learned_addr learned my addr 172.21.15.82:0/3739772495 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d973f700 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee726e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d8f3e700 1 -- 172.21.15.82:0/3739772495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 msgr2=0x558ee726e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee726e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d8f3e700 1 -- 172.21.15.82:0/3739772495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 msgr2=0x558ee72695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d8f3e700 1 -- 172.21.15.82:0/3739772495 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee6fe7d80 con 0x558ee726ecc0 2023-07-14T19:34:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.041+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7268c30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009990 tx=0x7f10c400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce2091503fcbe59 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.052+0000 7f10bf7fe700 1 -- 172.21.15.82:0/3739772495 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c402d030 con 0x558ee726ecc0 2023-07-14T19:34:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.052+0000 7f10bf7fe700 1 -- 172.21.15.82:0/3739772495 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c401e070 con 0x558ee726ecc0 2023-07-14T19:34:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.052+0000 7f10bf7fe700 1 -- 172.21.15.82:0/3739772495 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c40048c0 con 0x558ee726ecc0 2023-07-14T19:34:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3739772495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 msgr2=0x558ee7268c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7268c30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10c4009990 tx=0x7f10c400bdf0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3739772495 shutdown_connections 2023-07-14T19:34:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee726e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3739772495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7268c30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.053+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3739772495 >> 172.21.15.82:0/3739772495 conn(0x558ee725e620 msgr2=0x558ee7263660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:00.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.054+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f10c00a1400 0x7f10c00a38b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c010 tx=0x7f10c4004ef0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:00.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.089+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3739772495 shutdown_connections 2023-07-14T19:34:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.089+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3739772495 wait complete. 2023-07-14T19:34:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.090+0000 7f10e7b3bb80 1 -- start start 2023-07-14T19:34:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.090+0000 7f10e7b3bb80 1 Processor -- start 2023-07-14T19:34:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 0x558ee7275040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee7269170 0x558ee72722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7272a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7159a60 con 0x558ee70c46b0 2023-07-14T19:34:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7159760 con 0x558ee7269170 2023-07-14T19:34:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee703d8c0 con 0x558ee726ecc0 2023-07-14T19:34:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.091+0000 7f10e7b3bb80 10 client.0 _ll_register_callbacks cb 0x558ee7157700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 0x558ee7275040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 0x558ee7275040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44154/0 (socket says 172.21.15.82:44154) 2023-07-14T19:34:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 -- 172.21.15.82:0/3193235943 learned_addr learned my addr 172.21.15.82:0/3193235943 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 -- 172.21.15.82:0/3193235943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 msgr2=0x558ee7272a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T19:34:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7272a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 -- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee7269170 msgr2=0x558ee72722c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T19:34:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee7269170 0x558ee72722c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.092+0000 7f10d973f700 1 -- 172.21.15.82:0/3193235943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d411bf60 con 0x558ee70c46b0 2023-07-14T19:34:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.093+0000 7f10d973f700 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 0x558ee7275040 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00bec0 tx=0x7f10cc00a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5973c286182e4aa server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:00.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.093+0000 7f10bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:34:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.093+0000 7f10bcff9700 1 -- 172.21.15.82:0/3193235943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc016610 con 0x558ee70c46b0 2023-07-14T19:34:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.093+0000 7f10bcff9700 1 -- 172.21.15.82:0/3193235943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f10cc016770 con 0x558ee70c46b0 2023-07-14T19:34:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.094+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ee7113490 con 0x558ee70c46b0 2023-07-14T19:34:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.094+0000 7f10bcff9700 1 -- 172.21.15.82:0/3193235943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10cc00ae80 con 0x558ee70c46b0 2023-07-14T19:34:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.094+0000 7f10e7b3bb80 10 client.16011 fetch_fsmap learned FSMap version 36 2023-07-14T19:34:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.094+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558ee6fd0c00 con 0x558ee70c46b0 2023-07-14T19:34:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.095+0000 7f10bcff9700 1 -- 172.21.15.82:0/3193235943 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ee7113490 con 0x558ee70c46b0 2023-07-14T19:34:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.095+0000 7f10bcff9700 1 -- 172.21.15.82:0/3193235943 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f10cc015ae0 con 0x558ee70c46b0 2023-07-14T19:34:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.096+0000 7f10e7b3bb80 10 client.16011 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:34:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.096+0000 7f10e7b3bb80 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.096+0000 7f10e7b3bb80 20 client.16011 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.096+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72826b0 0x558ee7284ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.096+0000 7f10e7b3bb80 4 client.16011 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:34:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.096+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ee7237d40 con 0x558ee72826b0 2023-07-14T19:34:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.097+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72826b0 0x558ee7284ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.104+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72826b0 0x558ee7284ac0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde59e2de4541346 server_cookie=b53cafab7172512b in_seq=0 out_seq=0 2023-07-14T19:34:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.107+0000 7f10bcff9700 10 client.16011 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.107+0000 7f10bcff9700 1 -- 172.21.15.82:0/3193235943 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558ee7237d40 con 0x558ee72826b0 2023-07-14T19:34:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.107+0000 7f10bcff9700 10 client.16011 handle_command_reply: tid=0 2023-07-14T19:34:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 client.16011 shutdown 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 msgr2=0x558ee7275040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 0x558ee7275040 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00bec0 tx=0x7f10cc00a6c0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 shutdown_connections 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72826b0 0x558ee7284ac0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee7272a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee70c46b0 0x558ee7275040 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/3193235943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee7269170 0x558ee72722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.108+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 >> 172.21.15.82:0/3193235943 conn(0x558ee725e620 msgr2=0x558ee7262e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.109+0000 7f10bcff9700 0 client.16011 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.109+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 shutdown_connections 2023-07-14T19:34:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.109+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/3193235943 wait complete. 2023-07-14T19:34:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.114+0000 7f10e7b3bb80 20 client.16011 trim_cache size 0 max 16384 2023-07-14T19:34:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.168+0000 7f10e7b3bb80 1 Processor -- start 2023-07-14T19:34:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 -- start start 2023-07-14T19:34:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7268c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee7269170 0x558ee7279650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee726ecc0 0x558ee7263600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee727fc20 con 0x558ee726ecc0 2023-07-14T19:34:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7263b40 con 0x558ee70c46b0 2023-07-14T19:34:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.182+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee725b730 con 0x558ee7269170 2023-07-14T19:34:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.184+0000 7f10d973f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7268c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.184+0000 7f10d973f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7268c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40510/0 (socket says 172.21.15.82:40510) 2023-07-14T19:34:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.184+0000 7f10d973f700 1 -- 172.21.15.82:0/523163995 learned_addr learned my addr 172.21.15.82:0/523163995 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee7269170 0x558ee7279650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 -- 172.21.15.82:0/523163995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 msgr2=0x558ee7268c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7268c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 -- 172.21.15.82:0/523163995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee726ecc0 msgr2=0x558ee7263600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee726ecc0 0x558ee7263600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 -- 172.21.15.82:0/523163995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee700e1f0 con 0x558ee7269170 2023-07-14T19:34:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.185+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee7269170 0x558ee7279650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x558ee7273900 tx=0x7f10c402b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ada9100db622cf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.202+0000 7f10bffff700 1 -- 172.21.15.82:0/523163995 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c403a040 con 0x558ee7269170 2023-07-14T19:34:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.202+0000 7f10bffff700 1 -- 172.21.15.82:0/523163995 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c400ba80 con 0x558ee7269170 2023-07-14T19:34:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.202+0000 7f10bffff700 1 -- 172.21.15.82:0/523163995 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c402d040 con 0x558ee7269170 2023-07-14T19:34:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/523163995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee7269170 msgr2=0x558ee7279650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee7269170 0x558ee7279650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x558ee7273900 tx=0x7f10c402b000 comp rx=0 tx=0).stop 2023-07-14T19:34:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/523163995 shutdown_connections 2023-07-14T19:34:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee726ecc0 0x558ee7263600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7268c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/523163995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee7269170 0x558ee7279650 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.203+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/523163995 >> 172.21.15.82:0/523163995 conn(0x558ee725e620 msgr2=0x558ee727e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.276+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/523163995 shutdown_connections 2023-07-14T19:34:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.276+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/523163995 wait complete. 2023-07-14T19:34:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.277+0000 7f10e7b3bb80 1 -- start start 2023-07-14T19:34:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.277+0000 7f10e7b3bb80 1 Processor -- start 2023-07-14T19:34:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7275b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee72d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7276800 con 0x558ee7269170 2023-07-14T19:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7276970 con 0x558ee70c46b0 2023-07-14T19:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee72d5880 con 0x558ee726ecc0 2023-07-14T19:34:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.278+0000 7f10e7b3bb80 10 client.0 _ll_register_callbacks cb 0x7f10cc00b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.279+0000 7f10d8f3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.279+0000 7f10d8f3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44170/0 (socket says 172.21.15.82:44170) 2023-07-14T19:34:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.279+0000 7f10d8f3e700 1 -- 172.21.15.82:0/911265883 learned_addr learned my addr 172.21.15.82:0/911265883 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.279+0000 7f10d9f40700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee72d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.279+0000 7f10d973f700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7275b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.295+0000 7f10d8f3e700 1 -- 172.21.15.82:0/911265883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 msgr2=0x558ee72d5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.295+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee72d5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.295+0000 7f10d8f3e700 1 -- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 msgr2=0x558ee7275b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.295+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7275b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.295+0000 7f10d8f3e700 1 -- 172.21.15.82:0/911265883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ee6fe7d80 con 0x558ee7269170 2023-07-14T19:34:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.320+0000 7f10d8f3e700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72760b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f10c4011040 tx=0x7f10c4004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bf41efca0b4eb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.403+0000 7f10bcff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:34:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.403+0000 7f10bcff9700 1 -- 172.21.15.82:0/911265883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c402d030 con 0x558ee7269170 2023-07-14T19:34:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.403+0000 7f10bcff9700 1 -- 172.21.15.82:0/911265883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f10c402e070 con 0x558ee7269170 2023-07-14T19:34:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.403+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10c401e070 con 0x558ee7269170 2023-07-14T19:34:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.434+0000 7f10bcff9700 1 -- 172.21.15.82:0/911265883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10c403a040 con 0x558ee7269170 2023-07-14T19:34:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.436+0000 7f10e7b3bb80 10 client.16014 fetch_fsmap learned FSMap version 36 2023-07-14T19:34:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.436+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558ee6fe6250 con 0x558ee7269170 2023-07-14T19:34:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.440+0000 7f10bcff9700 1 -- 172.21.15.82:0/911265883 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c401e070 con 0x558ee7269170 2023-07-14T19:34:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10bcff9700 1 -- 172.21.15.82:0/911265883 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f10c404e070 con 0x558ee7269170 2023-07-14T19:34:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10e7b3bb80 10 client.16014 fetch_fsmap finished waiting for FSMap version 36 2023-07-14T19:34:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10e7b3bb80 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10e7b3bb80 20 client.16014 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72d6aa0 0x558ee72d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10e7b3bb80 4 client.16014 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:34:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.441+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ee71b2e50 con 0x558ee72d6aa0 2023-07-14T19:34:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.442+0000 7f10d973f700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72d6aa0 0x558ee72d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.442+0000 7f10d973f700 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72d6aa0 0x558ee72d8eb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937d19eaa66ee01d server_cookie=15e0eb076b491ec3 in_seq=0 out_seq=0 2023-07-14T19:34:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:00.458+0000 7f10bcff9700 10 client.16014 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.545+0000 7f10bcff9700 1 -- 172.21.15.82:0/911265883 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ee71b2e50 con 0x558ee72d6aa0 2023-07-14T19:34:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.545+0000 7f10bcff9700 10 client.16014 handle_command_reply: tid=0 2023-07-14T19:34:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.549+0000 7f10e7b3bb80 1 client.16014 shutdown 2023-07-14T19:34:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.550+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 msgr2=0x558ee72760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.550+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72760b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f10c4011040 tx=0x7f10c4004f60 comp rx=0 tx=0).stop 2023-07-14T19:34:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 shutdown_connections 2023-07-14T19:34:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558ee72d6aa0 0x558ee72d8eb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee726ecc0 0x558ee72d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558ee70c46b0 0x558ee7275b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 --2- 172.21.15.82:0/911265883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558ee7269170 0x558ee72760b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 >> 172.21.15.82:0/911265883 conn(0x558ee725e620 msgr2=0x558ee726c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10bcff9700 0 client.16014 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:01.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.551+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 shutdown_connections 2023-07-14T19:34:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:34:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:34:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:34:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:34:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.595+0000 7f10e7b3bb80 1 -- 172.21.15.82:0/911265883 wait complete. 2023-07-14T19:34:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.595+0000 7f10e7b3bb80 20 client.16014 trim_cache size 0 max 16384 2023-07-14T19:34:01.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:01 smithi133 ceph-mon[128894]: pgmap v167: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.5 MiB/s wr, 518 op/s 2023-07-14T19:34:01.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:01 smithi161 ceph-mon[129184]: pgmap v167: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.5 MiB/s wr, 518 op/s 2023-07-14T19:34:01.745 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:01 smithi082 ceph-mon[127461]: pgmap v167: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.5 MiB/s wr, 518 op/s 2023-07-14T19:34:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.806+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f10c00a1400 msgr2=0x7f10c00a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.806+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f10c00a1400 0x7f10c00a38b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c010 tx=0x7f10c4004ef0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.806+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d410e160 msgr2=0x7f10d41b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.806+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d410e160 0x7f10d41b8f90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10d000c7a0 tx=0x7f10d000cab0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 shutdown_connections 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f10c00a1400 0x7f10c00a38b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10d410e160 0x7f10d41b8f90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10d40712b0 0x7f10d4113dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 --2- 172.21.15.82:0/2591781264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10d4071c60 0x7f10d4114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 >> 172.21.15.82:0/2591781264 conn(0x7f10d406bf30 msgr2=0x7f10d410bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 shutdown_connections 2023-07-14T19:34:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:01.807+0000 7f10db9a3700 1 -- 172.21.15.82:0/2591781264 wait complete. 2023-07-14T19:34:02.217 INFO:tasks.workunit.client.0.smithi082.stdout: 332 2653181 379 1842726 514 1031782 1030 2023-07-14T19:34:02.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:02 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:02.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:02 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:02.766 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:02 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:02.983 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[158392]: *** Caught signal (Segmentation fault) ** 2023-07-14T19:34:02.984 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[158392]: in thread 7fd30e19a700 thread_name:MR_Finisher 2023-07-14T19:34:02.984 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:02 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-07-14T19:34:03.244 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:03 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Failed with result 'exit-code'. 2023-07-14T19:34:03.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:03 smithi133 ceph-mon[128894]: pgmap v168: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 453 op/s 2023-07-14T19:34:03.717 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:03 smithi082 ceph-mon[127461]: pgmap v168: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 453 op/s 2023-07-14T19:34:03.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:03 smithi161 ceph-mon[129184]: pgmap v168: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 453 op/s 2023-07-14T19:34:05.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:05 smithi133 ceph-mon[128894]: pgmap v169: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-07-14T19:34:05.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:05 smithi082 ceph-mon[127461]: pgmap v169: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-07-14T19:34:05.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:05 smithi161 ceph-mon[129184]: pgmap v169: 97 pgs: 97 active+clean; 1.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2023-07-14T19:34:07.686 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:07 smithi133 ceph-mon[128894]: pgmap v170: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 403 op/s 2023-07-14T19:34:07.724 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:07 smithi082 ceph-mon[127461]: pgmap v170: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 403 op/s 2023-07-14T19:34:07.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:07 smithi161 ceph-mon[129184]: pgmap v170: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 403 op/s 2023-07-14T19:34:09.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:09 smithi082 ceph-mon[127461]: pgmap v171: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 312 op/s 2023-07-14T19:34:09.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:09 smithi161 ceph-mon[129184]: pgmap v171: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 312 op/s 2023-07-14T19:34:09.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:09 smithi133 ceph-mon[128894]: pgmap v171: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 312 op/s 2023-07-14T19:34:10.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:10 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:34:10.721 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:10 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:34:10.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:10 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:34:10.744 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:10 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:34:10.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:10 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:34:10.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:10 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:34:10.936 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:10 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:34:10.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:10 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:34:10.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:10 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:34:11.718 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:11 smithi082 ceph-mon[127461]: pgmap v172: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 362 op/s 2023-07-14T19:34:11.745 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:11 smithi161 ceph-mon[129184]: pgmap v172: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 362 op/s 2023-07-14T19:34:11.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:11 smithi133 ceph-mon[128894]: pgmap v172: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 362 op/s 2023-07-14T19:34:12.219 INFO:tasks.workunit.client.0.smithi082.stdout: 341 2627281 578 1828014 556 1052421 1357 2023-07-14T19:34:13.501 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-07-14T19:34:13.501 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Scheduled restart job, restart counter is at 2. 2023-07-14T19:34:13.501 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 systemd[1]: Stopped Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:34:13.502 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 systemd[1]: Starting Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:34:13.799 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 podman[159728]: 2023-07-14T19:34:13.800 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:13 smithi161 ceph-mon[129184]: pgmap v173: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2023-07-14T19:34:13.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:13 smithi133 ceph-mon[128894]: pgmap v173: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2023-07-14T19:34:13.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:13 smithi082 ceph-mon[127461]: pgmap v173: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 479 op/s 2023-07-14T19:34:14.245 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 ceph-mds[159768]: -- 172.21.15.161:0/3481275151 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ea60e71e0 con 0x558ea60bc800 2023-07-14T19:34:14.245 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: starting mds.c at 2023-07-14T19:34:14.245 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 bash[159728]: 19b43656bed4fbf0be044c66fa087eff232396e0125978a6ec4d92f15fc8749f 2023-07-14T19:34:14.245 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:34:13 smithi161 systemd[1]: Started Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:34:14.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:14 smithi133 ceph-mon[128894]: pgmap v174: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 423 op/s 2023-07-14T19:34:14.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:14 smithi133 ceph-mon[128894]: daemon mds.c restarted 2023-07-14T19:34:14.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:14 smithi082 ceph-mon[127461]: pgmap v174: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 423 op/s 2023-07-14T19:34:14.969 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:14 smithi082 ceph-mon[127461]: daemon mds.c restarted 2023-07-14T19:34:14.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:14 smithi161 ceph-mon[129184]: pgmap v174: 97 pgs: 97 active+clean; 1.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 423 op/s 2023-07-14T19:34:14.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:14 smithi161 ceph-mon[129184]: daemon mds.c restarted 2023-07-14T19:34:15.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:15 smithi133 ceph-mon[128894]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T19:34:15.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:15 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6836/1886336093,v1:172.21.15.161:6837/1886336093] up:boot 2023-07-14T19:34:15.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:15 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:34:15.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:15 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:34:15.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:15 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:15.977 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:15 smithi082 ceph-mon[127461]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T19:34:15.978 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:15 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6836/1886336093,v1:172.21.15.161:6837/1886336093] up:boot 2023-07-14T19:34:15.978 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:15 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:34:15.978 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:15 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:34:15.978 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:15 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:15.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:15 smithi161 ceph-mon[129184]: osdmap e86: 12 total, 12 up, 12 in 2023-07-14T19:34:15.995 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:15 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6836/1886336093,v1:172.21.15.161:6837/1886336093] up:boot 2023-07-14T19:34:15.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:15 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:34:15.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:15 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:34:15.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:15 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:16.937 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:16 smithi133 ceph-mon[128894]: pgmap v176: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 543 op/s 2023-07-14T19:34:16.968 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:16 smithi082 ceph-mon[127461]: pgmap v176: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 543 op/s 2023-07-14T19:34:16.996 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:16 smithi161 ceph-mon[129184]: pgmap v176: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 543 op/s 2023-07-14T19:34:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.286+0000 7fb028800700 1 -- 172.21.15.82:0/745246202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006ca40 msgr2=0x7fb02006ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.286+0000 7fb028800700 1 --2- 172.21.15.82:0/745246202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006ca40 0x7fb02006ceb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009f90 tx=0x7fb01c00b330 comp rx=0 tx=0).stop 2023-07-14T19:34:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.311+0000 7fb028800700 1 -- 172.21.15.82:0/745246202 shutdown_connections 2023-07-14T19:34:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.311+0000 7fb028800700 1 --2- 172.21.15.82:0/745246202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb02006d4d0 0x7fb02011b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.311+0000 7fb028800700 1 --2- 172.21.15.82:0/745246202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02010e3d0 0x7fb02010e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.316 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.311+0000 7fb028800700 1 --2- 172.21.15.82:0/745246202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006ca40 0x7fb02006ceb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.311+0000 7fb028800700 1 -- 172.21.15.82:0/745246202 >> 172.21.15.82:0/745246202 conn(0x7fb02006bea0 msgr2=0x7fb020071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.316+0000 7fb028800700 1 -- 172.21.15.82:0/745246202 shutdown_connections 2023-07-14T19:34:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.317+0000 7fb028800700 1 -- 172.21.15.82:0/745246202 wait complete. 2023-07-14T19:34:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 Processor -- start 2023-07-14T19:34:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 -- start start 2023-07-14T19:34:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02006ca40 0x7fb0201aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 0x7fb0201ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb02010e3d0 0x7fb0201a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201a51b0 con 0x7fb02006d4d0 2023-07-14T19:34:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201a5320 con 0x7fb02006ca40 2023-07-14T19:34:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.318+0000 7fb028800700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0201a54e0 con 0x7fb02010e3d0 2023-07-14T19:34:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.320+0000 7fb025d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 0x7fb0201ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.320+0000 7fb025d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 0x7fb0201ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44182/0 (socket says 172.21.15.82:44182) 2023-07-14T19:34:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.320+0000 7fb025d9b700 1 -- 172.21.15.82:0/1763576914 learned_addr learned my addr 172.21.15.82:0/1763576914 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.320+0000 7fb026d9d700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb02010e3d0 0x7fb0201a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.322+0000 7fb025d9b700 1 -- 172.21.15.82:0/1763576914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb02010e3d0 msgr2=0x7fb0201a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.322+0000 7fb025d9b700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb02010e3d0 0x7fb0201a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.322+0000 7fb025d9b700 1 -- 172.21.15.82:0/1763576914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02006ca40 msgr2=0x7fb0201aace0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T19:34:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.322+0000 7fb025d9b700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02006ca40 0x7fb0201aace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.322+0000 7fb025d9b700 1 -- 172.21.15.82:0/1763576914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c009c40 con 0x7fb02006d4d0 2023-07-14T19:34:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.324+0000 7fb025d9b700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 0x7fb0201ab220 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb01c007760 tx=0x7fb01c007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.567+0000 7fb0137fe700 1 -- 172.21.15.82:0/1763576914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c00e040 con 0x7fb02006d4d0 2023-07-14T19:34:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.567+0000 7fb0137fe700 1 -- 172.21.15.82:0/1763576914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01c004e40 con 0x7fb02006d4d0 2023-07-14T19:34:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.567+0000 7fb0137fe700 1 -- 172.21.15.82:0/1763576914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c01b860 con 0x7fb02006d4d0 2023-07-14T19:34:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.573+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0201a5760 con 0x7fb02006d4d0 2023-07-14T19:34:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.574+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0201a59b0 con 0x7fb02006d4d0 2023-07-14T19:34:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.579+0000 7fb034998b80 1 Processor -- start 2023-07-14T19:34:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 -- start start 2023-07-14T19:34:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 0x55801cef0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0f00 0x55801ceeaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cdefc50 con 0x55801cd466b0 2023-07-14T19:34:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ce429e0 con 0x55801cef0a90 2023-07-14T19:34:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.580+0000 7fb034998b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ce3be70 con 0x55801cef0f00 2023-07-14T19:34:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.585+0000 7fb026d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 0x55801cef0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.585+0000 7fb026d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 0x55801cef0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44190/0 (socket says 172.21.15.82:44190) 2023-07-14T19:34:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.585+0000 7fb026d9d700 1 -- 172.21.15.82:0/2024745709 learned_addr learned my addr 172.21.15.82:0/2024745709 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.591+0000 7fb0137fe700 1 -- 172.21.15.82:0/1763576914 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb01c01daa0 con 0x7fb02006d4d0 2023-07-14T19:34:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.592+0000 7fb0137fe700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb00c0a14d0 0x7fb00c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.592+0000 7fb0137fe700 1 -- 172.21.15.82:0/1763576914 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fb01c0cd770 con 0x7fb02006d4d0 2023-07-14T19:34:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.594+0000 7fb025d9b700 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0f00 0x55801ceeaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.595+0000 7fb02659c700 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.597+0000 7fb026d9d700 1 -- 172.21.15.82:0/2024745709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0f00 msgr2=0x55801ceeaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.597+0000 7fb026d9d700 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0f00 0x55801ceeaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.597+0000 7fb026d9d700 1 -- 172.21.15.82:0/2024745709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 msgr2=0x55801ceea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.597+0000 7fb026d9d700 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.597+0000 7fb026d9d700 1 -- 172.21.15.82:0/2024745709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55801cc69d80 con 0x55801cd466b0 2023-07-14T19:34:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.608+0000 7fb026d9d700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb00c0a14d0 0x7fb00c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.610+0000 7fb026d9d700 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 0x55801cef0550 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d10 tx=0x7fb0180105f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf0f27dd3c286b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.611+0000 7fb010ff9700 1 -- 172.21.15.82:0/2024745709 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0180123f0 con 0x55801cd466b0 2023-07-14T19:34:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.611+0000 7fb010ff9700 1 -- 172.21.15.82:0/2024745709 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb018012a10 con 0x55801cd466b0 2023-07-14T19:34:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.611+0000 7fb010ff9700 1 -- 172.21.15.82:0/2024745709 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018019b00 con 0x55801cd466b0 2023-07-14T19:34:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 -- 172.21.15.82:0/2024745709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 msgr2=0x55801cef0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 0x55801cef0550 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d10 tx=0x7fb0180105f0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 -- 172.21.15.82:0/2024745709 shutdown_connections 2023-07-14T19:34:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0f00 0x55801ceeaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cd466b0 0x55801cef0550 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 --2- 172.21.15.82:0/2024745709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.612+0000 7fb034998b80 1 -- 172.21.15.82:0/2024745709 >> 172.21.15.82:0/2024745709 conn(0x55801cee03d0 msgr2=0x55801cee5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.617+0000 7fb026d9d700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb00c0a14d0 0x7fb00c0a3980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d10 tx=0x7fb0180105f0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:17.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.626+0000 7fb034998b80 1 -- 172.21.15.82:0/2024745709 shutdown_connections 2023-07-14T19:34:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.628+0000 7fb034998b80 1 -- 172.21.15.82:0/2024745709 wait complete. 2023-07-14T19:34:17.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.629+0000 7fb034998b80 1 -- start start 2023-07-14T19:34:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.629+0000 7fb034998b80 1 Processor -- start 2023-07-14T19:34:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.629+0000 7fb034998b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cd466b0 0x55801cef9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.630+0000 7fb034998b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0a90 0x55801cefa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.630+0000 7fb034998b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801cef4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.630+0000 7fb034998b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cddba60 con 0x55801cef0f00 2023-07-14T19:34:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.630+0000 7fb034998b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cddb760 con 0x55801cd466b0 2023-07-14T19:34:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.630+0000 7fb034998b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ccbf8c0 con 0x55801cef0a90 2023-07-14T19:34:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.630+0000 7fb034998b80 10 client.0 _ll_register_callbacks cb 0x55801cdd9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.633+0000 7fb026d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801cef4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.633+0000 7fb026d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801cef4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44196/0 (socket says 172.21.15.82:44196) 2023-07-14T19:34:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.633+0000 7fb026d9d700 1 -- 172.21.15.82:0/3542210861 learned_addr learned my addr 172.21.15.82:0/3542210861 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.634+0000 7fb02659c700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cd466b0 0x55801cef9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.640+0000 7fb025d9b700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0a90 0x55801cefa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.648+0000 7fb026d9d700 1 -- 172.21.15.82:0/3542210861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0a90 msgr2=0x55801cefa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.648+0000 7fb026d9d700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0a90 0x55801cefa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.648+0000 7fb026d9d700 1 -- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cd466b0 msgr2=0x55801cef9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.648+0000 7fb026d9d700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cd466b0 0x55801cef9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.648+0000 7fb026d9d700 1 -- 172.21.15.82:0/3542210861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018009970 con 0x55801cef0f00 2023-07-14T19:34:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.660+0000 7fb026d9d700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801cef4100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb0180098a0 tx=0x7fb018009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf0d062f8424853 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.738+0000 7fb009ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:34:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.738+0000 7fb009ffb700 1 -- 172.21.15.82:0/3542210861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018012800 con 0x55801cef0f00 2023-07-14T19:34:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.738+0000 7fb009ffb700 1 -- 172.21.15.82:0/3542210861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fb018016030 con 0x55801cef0f00 2023-07-14T19:34:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.738+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55801cd95490 con 0x55801cef0f00 2023-07-14T19:34:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.749+0000 7fb009ffb700 1 -- 172.21.15.82:0/3542210861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0180194a0 con 0x55801cef0f00 2023-07-14T19:34:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.749+0000 7fb034998b80 10 client.16026 fetch_fsmap learned FSMap version 39 2023-07-14T19:34:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.749+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55801cc52c00 con 0x55801cef0f00 2023-07-14T19:34:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.751+0000 7fb009ffb700 1 -- 172.21.15.82:0/3542210861 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55801cd95490 con 0x55801cef0f00 2023-07-14T19:34:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.751+0000 7fb009ffb700 1 -- 172.21.15.82:0/3542210861 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb018024440 con 0x55801cef0f00 2023-07-14T19:34:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.753+0000 7fb034998b80 10 client.16026 fetch_fsmap finished waiting for FSMap version 39 2023-07-14T19:34:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.753+0000 7fb034998b80 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.753+0000 7fb034998b80 20 client.16026 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.753+0000 7fb034998b80 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf150c0 0x55801cf174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.753+0000 7fb034998b80 4 client.16026 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:34:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.753+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55801ceb9d40 con 0x55801cf150c0 2023-07-14T19:34:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.763+0000 7fb02659c700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf150c0 0x55801cf174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.764+0000 7fb02659c700 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf150c0 0x55801cf174d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca620328a82fb63 server_cookie=3cd0af27ddf4db3e in_seq=0 out_seq=0 2023-07-14T19:34:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.764+0000 7fb009ffb700 10 client.16026 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.765+0000 7fb009ffb700 1 -- 172.21.15.82:0/3542210861 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55801ceb9d40 con 0x55801cf150c0 2023-07-14T19:34:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.765+0000 7fb009ffb700 10 client.16026 handle_command_reply: tid=0 2023-07-14T19:34:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.765+0000 7fb034998b80 1 client.16026 shutdown 2023-07-14T19:34:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.765+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 msgr2=0x55801cef4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.765+0000 7fb034998b80 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801cef4100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb0180098a0 tx=0x7fb018009610 comp rx=0 tx=0).stop 2023-07-14T19:34:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 shutdown_connections 2023-07-14T19:34:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801cef4100 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cd466b0 0x55801cef9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf150c0 0x55801cf174d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 --2- 172.21.15.82:0/3542210861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cef0a90 0x55801cefa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 >> 172.21.15.82:0/3542210861 conn(0x55801cee03d0 msgr2=0x55801cee4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb009ffb700 0 client.16026 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.766+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 shutdown_connections 2023-07-14T19:34:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.767+0000 7fb034998b80 1 -- 172.21.15.82:0/3542210861 wait complete. 2023-07-14T19:34:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:17.776+0000 7fb034998b80 20 client.16026 trim_cache size 0 max 16384 2023-07-14T19:34:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.044+0000 7fb034998b80 1 Processor -- start 2023-07-14T19:34:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 -- start start 2023-07-14T19:34:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801ceeaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ceeb780 con 0x55801cef0f00 2023-07-14T19:34:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ceb9d40 con 0x55801cef0a90 2023-07-14T19:34:18.050 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.045+0000 7fb034998b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cef4560 con 0x55801cd466b0 2023-07-14T19:34:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.054+0000 7fb025d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.054+0000 7fb02659c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.054+0000 7fb025d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40554/0 (socket says 172.21.15.82:40554) 2023-07-14T19:34:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.054+0000 7fb025d9b700 1 -- 172.21.15.82:0/853496793 learned_addr learned my addr 172.21.15.82:0/853496793 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.055+0000 7fb025d9b700 1 -- 172.21.15.82:0/853496793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 msgr2=0x55801cefa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.055+0000 7fb025d9b700 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.055+0000 7fb025d9b700 1 -- 172.21.15.82:0/853496793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 msgr2=0x55801ceeaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.055+0000 7fb025d9b700 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801ceeaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.055+0000 7fb025d9b700 1 -- 172.21.15.82:0/853496793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55801cc901f0 con 0x55801cef0a90 2023-07-14T19:34:18.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.067+0000 7fb025d9b700 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb01c096e00 tx=0x7fb01c0ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b6d97dc227c861 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.233+0000 7fb0117fa700 1 -- 172.21.15.82:0/853496793 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0b8ea0 con 0x55801cef0a90 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.233+0000 7fb0117fa700 1 -- 172.21.15.82:0/853496793 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb01c09e930 con 0x55801cef0a90 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.233+0000 7fb0117fa700 1 -- 172.21.15.82:0/853496793 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c09adc0 con 0x55801cef0a90 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.234+0000 7fb034998b80 1 -- 172.21.15.82:0/853496793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 msgr2=0x55801ceea9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.234+0000 7fb034998b80 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb01c096e00 tx=0x7fb01c0ba040 comp rx=0 tx=0).stop 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 -- 172.21.15.82:0/853496793 shutdown_connections 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0f00 0x55801ceeaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 --2- 172.21.15.82:0/853496793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0a90 0x55801ceea9e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 -- 172.21.15.82:0/853496793 >> 172.21.15.82:0/853496793 conn(0x55801cee03d0 msgr2=0x55801cef9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 -- 172.21.15.82:0/853496793 shutdown_connections 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.244+0000 7fb034998b80 1 -- 172.21.15.82:0/853496793 wait complete. 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 -- start start 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 Processor -- start 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0a90 0x55801cefd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0f00 0x55801cf67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cefd990 con 0x55801cef0a90 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cf68290 con 0x55801cef0f00 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801cf683d0 con 0x55801cd466b0 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.245+0000 7fb034998b80 10 client.0 _ll_register_callbacks cb 0x7fb018019390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.246+0000 7fb026d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0f00 0x55801cf67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.246+0000 7fb026d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0f00 0x55801cf67d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40560/0 (socket says 172.21.15.82:40560) 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.246+0000 7fb026d9d700 1 -- 172.21.15.82:0/3197959572 learned_addr learned my addr 172.21.15.82:0/3197959572 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.246+0000 7fb02659c700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.255+0000 7fb025d9b700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0a90 0x55801cefd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.255+0000 7fb02659c700 1 -- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0f00 msgr2=0x55801cf67d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.255+0000 7fb02659c700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0f00 0x55801cf67d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.255+0000 7fb02659c700 1 -- 172.21.15.82:0/3197959572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0a90 msgr2=0x55801cefd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.255+0000 7fb02659c700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0a90 0x55801cefd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.255+0000 7fb02659c700 1 -- 172.21.15.82:0/3197959572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0201a5760 con 0x55801cd466b0 2023-07-14T19:34:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.258+0000 7fb02659c700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefcb70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d7d0 tx=0x7fb01400df30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=702c54f61c4c2c6a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.348+0000 7fb009ffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:34:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.348+0000 7fb009ffb700 1 -- 172.21.15.82:0/3197959572 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014017040 con 0x55801cd466b0 2023-07-14T19:34:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.348+0000 7fb009ffb700 1 -- 172.21.15.82:0/3197959572 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fb014004250 con 0x55801cd466b0 2023-07-14T19:34:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.348+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb018012a10 con 0x55801cd466b0 2023-07-14T19:34:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.361+0000 7fb009ffb700 1 -- 172.21.15.82:0/3197959572 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb014004450 con 0x55801cd466b0 2023-07-14T19:34:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.361+0000 7fb034998b80 10 client.25573 fetch_fsmap learned FSMap version 39 2023-07-14T19:34:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.361+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55801cc68250 con 0x55801cd466b0 2023-07-14T19:34:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb009ffb700 1 -- 172.21.15.82:0/3197959572 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb018012a10 con 0x55801cd466b0 2023-07-14T19:34:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb009ffb700 1 -- 172.21.15.82:0/3197959572 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 39) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb014012510 con 0x55801cd466b0 2023-07-14T19:34:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb034998b80 10 client.25573 fetch_fsmap finished waiting for FSMap version 39 2023-07-14T19:34:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb034998b80 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb034998b80 20 client.25573 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb034998b80 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf69b20 0x55801cf6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb034998b80 4 client.25573 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:34:18.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.364+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55801ce34e50 con 0x55801cf69b20 2023-07-14T19:34:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.370+0000 7fb025d9b700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf69b20 0x55801cf6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.402+0000 7fb025d9b700 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf69b20 0x55801cf6bf30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68dfbc7ece4e842 server_cookie=6d6cd99654d438a in_seq=0 out_seq=0 2023-07-14T19:34:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:18.402+0000 7fb009ffb700 10 client.25573 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:19.220 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:18 smithi082 ceph-mon[127461]: pgmap v177: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 543 op/s 2023-07-14T19:34:19.247 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:18 smithi161 ceph-mon[129184]: pgmap v177: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 543 op/s 2023-07-14T19:34:19.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:18 smithi133 ceph-mon[128894]: pgmap v177: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 543 op/s 2023-07-14T19:34:21.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:21 smithi133 ceph-mon[128894]: pgmap v178: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 646 op/s 2023-07-14T19:34:21.470 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:21 smithi082 ceph-mon[127461]: pgmap v178: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 646 op/s 2023-07-14T19:34:21.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:21 smithi161 ceph-mon[129184]: pgmap v178: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.4 MiB/s wr, 646 op/s 2023-07-14T19:34:22.227 INFO:tasks.workunit.client.0.smithi082.stdout: 347 2397190 620 1669563 226 974069 1029 2023-07-14T19:34:23.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:23 smithi133 ceph-mon[128894]: pgmap v179: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 707 op/s 2023-07-14T19:34:23.472 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:23 smithi082 ceph-mon[127461]: pgmap v179: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 707 op/s 2023-07-14T19:34:23.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:23 smithi161 ceph-mon[129184]: pgmap v179: 97 pgs: 97 active+clean; 1.4 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 707 op/s 2023-07-14T19:34:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.616+0000 7fb009ffb700 1 -- 172.21.15.82:0/3197959572 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55801ce34e50 con 0x55801cf69b20 2023-07-14T19:34:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.616+0000 7fb009ffb700 10 client.25573 handle_command_reply: tid=0 2023-07-14T19:34:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.617+0000 7fb034998b80 1 client.25573 shutdown 2023-07-14T19:34:24.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 msgr2=0x55801cefcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefcb70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d7d0 tx=0x7fb01400df30 comp rx=0 tx=0).stop 2023-07-14T19:34:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 shutdown_connections 2023-07-14T19:34:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55801cef0f00 0x55801cf67d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55801cf69b20 0x55801cf6bf30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55801cd466b0 0x55801cefcb70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 --2- 172.21.15.82:0/3197959572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55801cef0a90 0x55801cefd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 >> 172.21.15.82:0/3197959572 conn(0x55801cee03d0 msgr2=0x55801ceef1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.621+0000 7fb009ffb700 0 client.25573 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.622+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 shutdown_connections 2023-07-14T19:34:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.622+0000 7fb034998b80 1 -- 172.21.15.82:0/3197959572 wait complete. 2023-07-14T19:34:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.623+0000 7fb034998b80 20 client.25573 trim_cache size 0 max 16384 2023-07-14T19:34:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:34:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:34:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:34:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:34:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.654+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb00c0a14d0 msgr2=0x7fb00c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.654+0000 7fb028800700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb00c0a14d0 0x7fb00c0a3980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb018009d10 tx=0x7fb0180105f0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.654+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 msgr2=0x7fb0201ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.654+0000 7fb028800700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 0x7fb0201ab220 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb01c007760 tx=0x7fb01c007790 comp rx=0 tx=0).stop 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.665+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 shutdown_connections 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.665+0000 7fb028800700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb00c0a14d0 0x7fb00c0a3980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.665+0000 7fb028800700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb02010e3d0 0x7fb0201a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.665+0000 7fb028800700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb02006ca40 0x7fb0201aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.665+0000 7fb028800700 1 --2- 172.21.15.82:0/1763576914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02006d4d0 0x7fb0201ab220 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.665+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 >> 172.21.15.82:0/1763576914 conn(0x7fb02006bea0 msgr2=0x7fb02006f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.686+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 shutdown_connections 2023-07-14T19:34:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:24.697+0000 7fb028800700 1 -- 172.21.15.82:0/1763576914 wait complete. 2023-07-14T19:34:25.437 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:25 smithi133 ceph-mon[128894]: pgmap v180: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 924 op/s 2023-07-14T19:34:25.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:25 smithi082 ceph-mon[127461]: pgmap v180: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 924 op/s 2023-07-14T19:34:25.495 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:25 smithi161 ceph-mon[129184]: pgmap v180: 97 pgs: 97 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.1 MiB/s wr, 924 op/s 2023-07-14T19:34:27.438 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:27 smithi133 ceph-mon[128894]: pgmap v181: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-07-14T19:34:27.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:27 smithi082 ceph-mon[127461]: pgmap v181: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-07-14T19:34:27.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:27 smithi161 ceph-mon[129184]: pgmap v181: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-07-14T19:34:29.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:29 smithi082 ceph-mon[127461]: pgmap v182: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 910 op/s 2023-07-14T19:34:29.496 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:29 smithi161 ceph-mon[129184]: pgmap v182: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 910 op/s 2023-07-14T19:34:29.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:29 smithi133 ceph-mon[128894]: pgmap v182: 97 pgs: 97 active+clean; 1.5 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 11 MiB/s wr, 910 op/s 2023-07-14T19:34:31.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:31 smithi133 ceph-mon[128894]: pgmap v183: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T19:34:31.722 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:31 smithi082 ceph-mon[127461]: pgmap v183: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T19:34:31.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:31 smithi161 ceph-mon[129184]: pgmap v183: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-07-14T19:34:32.242 INFO:tasks.workunit.client.0.smithi082.stdout: 353 2227868 432 1539260 301 908895 1070 2023-07-14T19:34:32.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:32 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:32.724 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:32 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:32.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:32 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:34:33.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:33 smithi133 ceph-mon[128894]: pgmap v184: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 975 op/s 2023-07-14T19:34:33.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:33 smithi082 ceph-mon[127461]: pgmap v184: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 975 op/s 2023-07-14T19:34:33.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:33 smithi161 ceph-mon[129184]: pgmap v184: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 975 op/s 2023-07-14T19:34:35.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:35 smithi133 ceph-mon[128894]: pgmap v185: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 792 op/s 2023-07-14T19:34:35.725 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:35 smithi082 ceph-mon[127461]: pgmap v185: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 792 op/s 2023-07-14T19:34:35.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:35 smithi161 ceph-mon[129184]: pgmap v185: 97 pgs: 97 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 792 op/s 2023-07-14T19:34:37.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:37 smithi133 ceph-mon[128894]: pgmap v186: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 678 op/s 2023-07-14T19:34:37.723 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:37 smithi082 ceph-mon[127461]: pgmap v186: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 678 op/s 2023-07-14T19:34:37.746 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:37 smithi161 ceph-mon[129184]: pgmap v186: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 678 op/s 2023-07-14T19:34:39.688 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:39 smithi133 ceph-mon[128894]: pgmap v187: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.8 MiB/s wr, 420 op/s 2023-07-14T19:34:39.726 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:39 smithi082 ceph-mon[127461]: pgmap v187: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.8 MiB/s wr, 420 op/s 2023-07-14T19:34:39.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:39 smithi161 ceph-mon[129184]: pgmap v187: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 6.8 MiB/s wr, 420 op/s 2023-07-14T19:34:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.385+0000 7f3165bc4700 1 -- 172.21.15.82:0/3570316607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316010e3d0 msgr2=0x7f316010e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.385+0000 7f3165bc4700 1 --2- 172.21.15.82:0/3570316607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316010e3d0 0x7f316010e7a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f315800b0a0 tx=0x7f315800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 -- 172.21.15.82:0/3570316607 shutdown_connections 2023-07-14T19:34:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 --2- 172.21.15.82:0/3570316607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 0x7f316011b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 --2- 172.21.15.82:0/3570316607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316010e3d0 0x7f316010e7a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 --2- 172.21.15.82:0/3570316607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316006ca40 0x7f316006ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 -- 172.21.15.82:0/3570316607 >> 172.21.15.82:0/3570316607 conn(0x7f316006bea0 msgr2=0x7f3160071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 -- 172.21.15.82:0/3570316607 shutdown_connections 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.386+0000 7f3165bc4700 1 -- 172.21.15.82:0/3570316607 wait complete. 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.387+0000 7f3165bc4700 1 Processor -- start 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.387+0000 7f3165bc4700 1 -- start start 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.388+0000 7f3165bc4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316006ca40 0x7f31601a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.388+0000 7f3165bc4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 0x7f31601a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.388+0000 7f3165bc4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316010e3d0 0x7f31601a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.388+0000 7f3165bc4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601a9f10 con 0x7f316006ca40 2023-07-14T19:34:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.388+0000 7f3165bc4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601aa080 con 0x7f316006d4d0 2023-07-14T19:34:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.388+0000 7f3165bc4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601aa240 con 0x7f316010e3d0 2023-07-14T19:34:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.389+0000 7f315ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316010e3d0 0x7f31601a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.389+0000 7f315ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316010e3d0 0x7f31601a98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53490/0 (socket says 172.21.15.82:53490) 2023-07-14T19:34:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.389+0000 7f315ffff700 1 -- 172.21.15.82:0/1940674237 learned_addr learned my addr 172.21.15.82:0/1940674237 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.389+0000 7f315effd700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 0x7f31601a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.399+0000 7f315f7fe700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316006ca40 0x7f31601a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.399+0000 7f315effd700 1 -- 172.21.15.82:0/1940674237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316010e3d0 msgr2=0x7f31601a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.399+0000 7f315effd700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316010e3d0 0x7f31601a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.399+0000 7f315effd700 1 -- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316006ca40 msgr2=0x7f31601a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.399+0000 7f315effd700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316006ca40 0x7f31601a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.399+0000 7f315effd700 1 -- 172.21.15.82:0/1940674237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158009d00 con 0x7f316006d4d0 2023-07-14T19:34:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.400+0000 7f315effd700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 0x7f31601a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f315000d950 tx=0x7f315000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.634+0000 7f315cff9700 1 -- 172.21.15.82:0/1940674237 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150009930 con 0x7f316006d4d0 2023-07-14T19:34:41.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.634+0000 7f315cff9700 1 -- 172.21.15.82:0/1940674237 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3150010460 con 0x7f316006d4d0 2023-07-14T19:34:41.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.634+0000 7f315cff9700 1 -- 172.21.15.82:0/1940674237 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315000b5d0 con 0x7f316006d4d0 2023-07-14T19:34:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.634+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31601aa520 con 0x7f316006d4d0 2023-07-14T19:34:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.634+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316010ef50 con 0x7f316006d4d0 2023-07-14T19:34:41.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.643+0000 7f315cff9700 1 -- 172.21.15.82:0/1940674237 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f315000f620 con 0x7f316006d4d0 2023-07-14T19:34:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.644+0000 7f315cff9700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f31480a14d0 0x7f31480a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.644+0000 7f315cff9700 1 -- 172.21.15.82:0/1940674237 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f31500cd7d0 con 0x7f316006d4d0 2023-07-14T19:34:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.646+0000 7f315f7fe700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f31480a14d0 0x7f31480a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.653+0000 7f3171d5cb80 1 Processor -- start 2023-07-14T19:34:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.653+0000 7f3171d5cb80 1 -- start start 2023-07-14T19:34:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.653+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3ced550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.654+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3ce79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.654+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 0x5576e3ce7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.654+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3becc50 con 0x5576e3cedf00 2023-07-14T19:34:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.654+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3c3f9e0 con 0x5576e3b436b0 2023-07-14T19:34:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.654+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3c38e70 con 0x5576e3ceda90 2023-07-14T19:34:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.666+0000 7f315effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3ce79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.667+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3ced550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.671+0000 7f315ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 0x5576e3ce7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.671+0000 7f315ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 0x5576e3ce7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44232/0 (socket says 172.21.15.82:44232) 2023-07-14T19:34:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.671+0000 7f315ffff700 1 -- 172.21.15.82:0/581697418 learned_addr learned my addr 172.21.15.82:0/581697418 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.683+0000 7f315ffff700 1 -- 172.21.15.82:0/581697418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 msgr2=0x5576e3ce79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.683+0000 7f315ffff700 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3ce79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.683+0000 7f315ffff700 1 -- 172.21.15.82:0/581697418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 msgr2=0x5576e3ced550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.683+0000 7f315ffff700 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3ced550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.683+0000 7f315ffff700 1 -- 172.21.15.82:0/581697418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e3a66d80 con 0x5576e3cedf00 2023-07-14T19:34:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.685+0000 7f315f7fe700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f31480a14d0 0x7f31480a3980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f315800b780 tx=0x7f315800bab0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.693+0000 7f315ffff700 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 0x5576e3ce7f80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3154009da0 tx=0x7f3154011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6407d5c2ecd5a60 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.701+0000 7f3145ffb700 1 -- 172.21.15.82:0/581697418 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154009920 con 0x5576e3cedf00 2023-07-14T19:34:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.701+0000 7f3145ffb700 1 -- 172.21.15.82:0/581697418 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154019af0 con 0x5576e3cedf00 2023-07-14T19:34:41.710 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.701+0000 7f3145ffb700 1 -- 172.21.15.82:0/581697418 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315400fc40 con 0x5576e3cedf00 2023-07-14T19:34:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.702+0000 7f3171d5cb80 1 -- 172.21.15.82:0/581697418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 msgr2=0x5576e3ce7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.702+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 0x5576e3ce7f80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3154009da0 tx=0x7f3154011720 comp rx=0 tx=0).stop 2023-07-14T19:34:41.727 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:41 smithi082 ceph-mon[127461]: pgmap v188: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 351 op/s 2023-07-14T19:34:41.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:41 smithi161 ceph-mon[129184]: pgmap v188: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 351 op/s 2023-07-14T19:34:41.938 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:41 smithi133 ceph-mon[128894]: pgmap v188: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 351 op/s 2023-07-14T19:34:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 -- 172.21.15.82:0/581697418 shutdown_connections 2023-07-14T19:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3cedf00 0x5576e3ce7f80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3ced550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/581697418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3ce79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 -- 172.21.15.82:0/581697418 >> 172.21.15.82:0/581697418 conn(0x5576e3cdd3d0 msgr2=0x5576e3ce2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 -- 172.21.15.82:0/581697418 shutdown_connections 2023-07-14T19:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.937+0000 7f3171d5cb80 1 -- 172.21.15.82:0/581697418 wait complete. 2023-07-14T19:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.938+0000 7f3171d5cb80 1 -- start start 2023-07-14T19:34:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.938+0000 7f3171d5cb80 1 Processor -- start 2023-07-14T19:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.938+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cf10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3bd8a60 con 0x5576e3b436b0 2023-07-14T19:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3bd8760 con 0x5576e3cedf00 2023-07-14T19:34:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3abc8c0 con 0x5576e3ceda90 2023-07-14T19:34:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f3171d5cb80 10 client.0 _ll_register_callbacks cb 0x5576e3bd6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f315effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cf10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f315effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cf10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53502/0 (socket says 172.21.15.82:53502) 2023-07-14T19:34:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f315effd700 1 -- 172.21.15.82:0/937802417 learned_addr learned my addr 172.21.15.82:0/937802417 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.939+0000 7f315ffff700 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.940+0000 7f315effd700 1 -- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 msgr2=0x5576e3cf17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.940+0000 7f315effd700 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.940+0000 7f315effd700 1 -- 172.21.15.82:0/937802417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 msgr2=0x5576e3cf3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.940+0000 7f315effd700 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.940+0000 7f315effd700 1 -- 172.21.15.82:0/937802417 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3154009a80 con 0x5576e3ceda90 2023-07-14T19:34:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:41.940+0000 7f315effd700 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cf10d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f315009a5d0 tx=0x7f315009a9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce1df506e932b3c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.109+0000 7f31437f6700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:34:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.109+0000 7f31437f6700 1 -- 172.21.15.82:0/937802417 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150099dc0 con 0x5576e3ceda90 2023-07-14T19:34:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.109+0000 7f31437f6700 1 -- 172.21.15.82:0/937802417 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f3150099f20 con 0x5576e3ceda90 2023-07-14T19:34:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.109+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576e3b92490 con 0x5576e3ceda90 2023-07-14T19:34:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.122+0000 7f31437f6700 1 -- 172.21.15.82:0/937802417 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31500a2890 con 0x5576e3ceda90 2023-07-14T19:34:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.131+0000 7f3171d5cb80 10 client.25585 fetch_fsmap learned FSMap version 40 2023-07-14T19:34:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.131+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5576e3a4fc00 con 0x5576e3ceda90 2023-07-14T19:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.134+0000 7f31437f6700 1 -- 172.21.15.82:0/937802417 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576e3b92490 con 0x5576e3ceda90 2023-07-14T19:34:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.134+0000 7f31437f6700 1 -- 172.21.15.82:0/937802417 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f31500b4070 con 0x5576e3ceda90 2023-07-14T19:34:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.138+0000 7f3171d5cb80 10 client.25585 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T19:34:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.138+0000 7f3171d5cb80 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.138+0000 7f3171d5cb80 20 client.25585 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:42.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.138+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d01340 0x5576e3d03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.138+0000 7f3171d5cb80 4 client.25585 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:34:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.138+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576e3cb6d40 con 0x5576e3d01340 2023-07-14T19:34:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.152+0000 7f315f7fe700 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d01340 0x5576e3d03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.158+0000 7f315f7fe700 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d01340 0x5576e3d03750 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2483056a19a57764 server_cookie=9c78aa0ef9e94161 in_seq=0 out_seq=0 2023-07-14T19:34:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.158+0000 7f31437f6700 10 client.25585 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.163+0000 7f31437f6700 1 -- 172.21.15.82:0/937802417 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5576e3cb6d40 con 0x5576e3d01340 2023-07-14T19:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.163+0000 7f31437f6700 10 client.25585 handle_command_reply: tid=0 2023-07-14T19:34:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.163+0000 7f3171d5cb80 1 client.25585 shutdown 2023-07-14T19:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.163+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 msgr2=0x5576e3cf10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.163+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cf10d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f315009a5d0 tx=0x7f315009a9c0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 shutdown_connections 2023-07-14T19:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d01340 0x5576e3d03750 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/937802417 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cf10d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 >> 172.21.15.82:0/937802417 conn(0x5576e3cdd3d0 msgr2=0x5576e3ce1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f31437f6700 0 client.25585 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 shutdown_connections 2023-07-14T19:34:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.164+0000 7f3171d5cb80 1 -- 172.21.15.82:0/937802417 wait complete. 2023-07-14T19:34:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.165+0000 7f3171d5cb80 20 client.25585 trim_cache size 0 max 16384 2023-07-14T19:34:42.239 INFO:tasks.workunit.client.0.smithi082.stdout: 359 2672556 406 1839468 486 1119409 1286 2023-07-14T19:34:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 Processor -- start 2023-07-14T19:34:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 -- start start 2023-07-14T19:34:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3cdf490 con 0x5576e3b436b0 2023-07-14T19:34:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3cf2110 con 0x5576e3cedf00 2023-07-14T19:34:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.237+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3ce3330 con 0x5576e3ceda90 2023-07-14T19:34:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44246/0 (socket says 172.21.15.82:44246) 2023-07-14T19:34:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 -- 172.21.15.82:0/4072729991 learned_addr learned my addr 172.21.15.82:0/4072729991 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 -- 172.21.15.82:0/4072729991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 msgr2=0x5576e3cec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 -- 172.21.15.82:0/4072729991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 msgr2=0x5576e3cf0660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T19:34:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf0660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 -- 172.21.15.82:0/4072729991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576e3a8d1f0 con 0x5576e3b436b0 2023-07-14T19:34:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.238+0000 7f315f7fe700 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf8490 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3158009fd0 tx=0x7f315801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b08009b48ceb87 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.239+0000 7f31467fc700 1 -- 172.21.15.82:0/4072729991 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315800e040 con 0x5576e3b436b0 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.239+0000 7f31467fc700 1 -- 172.21.15.82:0/4072729991 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3158007960 con 0x5576e3b436b0 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.239+0000 7f31467fc700 1 -- 172.21.15.82:0/4072729991 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315802bd90 con 0x5576e3b436b0 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 -- 172.21.15.82:0/4072729991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 msgr2=0x5576e3cf8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf8490 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3158009fd0 tx=0x7f315801e000 comp rx=0 tx=0).stop 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 -- 172.21.15.82:0/4072729991 shutdown_connections 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3cedf00 0x5576e3cf0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3b436b0 0x5576e3cf8490 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/4072729991 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3ceda90 0x5576e3cec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 -- 172.21.15.82:0/4072729991 >> 172.21.15.82:0/4072729991 conn(0x5576e3cdd3d0 msgr2=0x5576e3cec140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 -- 172.21.15.82:0/4072729991 shutdown_connections 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 -- 172.21.15.82:0/4072729991 wait complete. 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.240+0000 7f3171d5cb80 1 -- start start 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 Processor -- start 2023-07-14T19:34:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3cea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3ceda90 0x5576e3ce8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3cedf00 0x5576e3ce9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3ce9bd0 con 0x5576e3ceda90 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3ce9d40 con 0x5576e3b436b0 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576e3d54fe0 con 0x5576e3cedf00 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f3171d5cb80 10 client.0 _ll_register_callbacks cb 0x5576e3bd6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3cea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f315f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3cea7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40600/0 (socket says 172.21.15.82:40600) 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f315f7fe700 1 -- 172.21.15.82:0/1087664006 learned_addr learned my addr 172.21.15.82:0/1087664006 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f315ffff700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3cedf00 0x5576e3ce9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.241+0000 7f315effd700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3ceda90 0x5576e3ce8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f315f7fe700 1 -- 172.21.15.82:0/1087664006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3cedf00 msgr2=0x5576e3ce9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f315f7fe700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3cedf00 0x5576e3ce9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f315f7fe700 1 -- 172.21.15.82:0/1087664006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3ceda90 msgr2=0x5576e3ce8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f315f7fe700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3ceda90 0x5576e3ce8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f315f7fe700 1 -- 172.21.15.82:0/1087664006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316011c0e0 con 0x5576e3b436b0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f315f7fe700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3cea7f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3158009610 tx=0x7f3158007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b476e413025a0e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f31437f6700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f31437f6700 1 -- 172.21.15.82:0/1087664006 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315800e040 con 0x5576e3b436b0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f31437f6700 1 -- 172.21.15.82:0/1087664006 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f3158039070 con 0x5576e3b436b0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3154019af0 con 0x5576e3b436b0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f31437f6700 1 -- 172.21.15.82:0/1087664006 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f315802b460 con 0x5576e3b436b0 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f3171d5cb80 10 client.25745 fetch_fsmap learned FSMap version 40 2023-07-14T19:34:42.255 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5576e3a65250 con 0x5576e3b436b0 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.242+0000 7f31437f6700 1 -- 172.21.15.82:0/1087664006 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3154019af0 con 0x5576e3b436b0 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f31437f6700 1 -- 172.21.15.82:0/1087664006 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f315803ea20 con 0x5576e3b436b0 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f3171d5cb80 10 client.25745 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f3171d5cb80 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f3171d5cb80 20 client.25745 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d56160 0x5576e3d58540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f3171d5cb80 4 client.25745 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:34:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576e3c31e50 con 0x5576e3d56160 2023-07-14T19:34:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f315effd700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d56160 0x5576e3d58540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f31437f6700 10 client.25745 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:42.243+0000 7f315effd700 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d56160 0x5576e3d58540 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99194475af5894b2 server_cookie=2e011f0a5f360b06 in_seq=0 out_seq=0 2023-07-14T19:34:43.719 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:43 smithi082 ceph-mon[127461]: pgmap v189: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 232 op/s 2023-07-14T19:34:43.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:43 smithi161 ceph-mon[129184]: pgmap v189: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 232 op/s 2023-07-14T19:34:43.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:43 smithi133 ceph-mon[128894]: pgmap v189: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 232 op/s 2023-07-14T19:34:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.971+0000 7f31437f6700 1 -- 172.21.15.82:0/1087664006 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576e3c31e50 con 0x5576e3d56160 2023-07-14T19:34:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.971+0000 7f31437f6700 10 client.25745 handle_command_reply: tid=0 2023-07-14T19:34:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 client.25745 shutdown 2023-07-14T19:34:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 msgr2=0x5576e3cea7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3cea7f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3158009610 tx=0x7f3158007690 comp rx=0 tx=0).stop 2023-07-14T19:34:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 shutdown_connections 2023-07-14T19:34:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5576e3d56160 0x5576e3d58540 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.980 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f31437f6700 0 client.25745 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5576e3cedf00 0x5576e3ce9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576e3b436b0 0x5576e3cea7f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 --2- 172.21.15.82:0/1087664006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5576e3ceda90 0x5576e3ce8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 >> 172.21.15.82:0/1087664006 conn(0x5576e3cdd3d0 msgr2=0x5576e3cec9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 shutdown_connections 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 1 -- 172.21.15.82:0/1087664006 wait complete. 2023-07-14T19:34:43.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.972+0000 7f3171d5cb80 20 client.25745 trim_cache size 0 max 16384 2023-07-14T19:34:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:34:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:34:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:34:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:34:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f31480a14d0 msgr2=0x7f31480a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f31480a14d0 0x7f31480a3980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f315800b780 tx=0x7f315800bab0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 msgr2=0x7f31601a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 0x7f31601a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f315000d950 tx=0x7f315000dc60 comp rx=0 tx=0).stop 2023-07-14T19:34:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 shutdown_connections 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f31480a14d0 0x7f31480a3980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f316010e3d0 0x7f31601a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f316006ca40 0x7f31601a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 --2- 172.21.15.82:0/1940674237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f316006d4d0 0x7f31601a5260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 >> 172.21.15.82:0/1940674237 conn(0x7f316006bea0 msgr2=0x7f31600708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 shutdown_connections 2023-07-14T19:34:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:43.973+0000 7f3165bc4700 1 -- 172.21.15.82:0/1940674237 wait complete. 2023-07-14T19:34:44.625 INFO:tasks.workunit.client.0.smithi082.stdout: 2023-07-14T19:34:44.625 INFO:tasks.workunit.client.0.smithi082.stdout:Final score for writes: 359 2023-07-14T19:34:44.626 INFO:tasks.workunit.client.0.smithi082.stdout:Final score for reads : 1033488 2023-07-14T19:34:44.626 INFO:tasks.workunit.client.0.smithi082.stdout: 2023-07-14T19:34:44.626 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-14T19:34:44.626 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-14T19:34:45.720 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:45 smithi082 ceph-mon[127461]: pgmap v190: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 360 op/s 2023-07-14T19:34:45.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:45 smithi161 ceph-mon[129184]: pgmap v190: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 360 op/s 2023-07-14T19:34:45.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:45 smithi133 ceph-mon[128894]: pgmap v190: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 360 op/s 2023-07-14T19:34:47.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:47 smithi161 ceph-mon[129184]: pgmap v191: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 917 op/s 2023-07-14T19:34:47.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:47 smithi133 ceph-mon[128894]: pgmap v191: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 917 op/s 2023-07-14T19:34:47.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:47 smithi082 ceph-mon[127461]: pgmap v191: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 917 op/s 2023-07-14T19:34:49.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:49 smithi161 ceph-mon[129184]: pgmap v192: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 1.00k op/s 2023-07-14T19:34:49.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:49 smithi133 ceph-mon[128894]: pgmap v192: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 1.00k op/s 2023-07-14T19:34:49.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:49 smithi082 ceph-mon[127461]: pgmap v192: 97 pgs: 97 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 1.00k op/s 2023-07-14T19:34:51.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:51 smithi161 ceph-mon[129184]: pgmap v193: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 1.02k op/s 2023-07-14T19:34:51.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:51 smithi133 ceph-mon[128894]: pgmap v193: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 1.02k op/s 2023-07-14T19:34:51.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:51 smithi082 ceph-mon[127461]: pgmap v193: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 1.02k op/s 2023-07-14T19:34:53.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:53 smithi161 ceph-mon[129184]: pgmap v194: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 1.10k op/s 2023-07-14T19:34:53.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:53 smithi133 ceph-mon[128894]: pgmap v194: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 1.10k op/s 2023-07-14T19:34:53.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:53 smithi082 ceph-mon[127461]: pgmap v194: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 1.10k op/s 2023-07-14T19:34:55.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:55 smithi161 ceph-mon[129184]: pgmap v195: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 1.23k op/s 2023-07-14T19:34:55.939 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:55 smithi133 ceph-mon[128894]: pgmap v195: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 1.23k op/s 2023-07-14T19:34:55.970 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:55 smithi082 ceph-mon[127461]: pgmap v195: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 1.23k op/s 2023-07-14T19:34:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 -- 172.21.15.82:0/3980775341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec700ff7c0 msgr2=0x7fec70103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 --2- 172.21.15.82:0/3980775341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec700ff7c0 0x7fec70103780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fec640099c0 tx=0x7fec64009cd0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 -- 172.21.15.82:0/3980775341 shutdown_connections 2023-07-14T19:34:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 --2- 172.21.15.82:0/3980775341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70103e60 0x7fec7010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 --2- 172.21.15.82:0/3980775341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec700fee20 0x7fec700ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 --2- 172.21.15.82:0/3980775341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec700ff7c0 0x7fec70103780 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 -- 172.21.15.82:0/3980775341 >> 172.21.15.82:0/3980775341 conn(0x7fec70076ae0 msgr2=0x7fec70076ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 -- 172.21.15.82:0/3980775341 shutdown_connections 2023-07-14T19:34:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.081+0000 7fec7621b700 1 -- 172.21.15.82:0/3980775341 wait complete. 2023-07-14T19:34:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 Processor -- start 2023-07-14T19:34:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 -- start start 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec700fee20 0x7fec70195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec700ff7c0 0x7fec701962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 0x7fec7019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7019af50 con 0x7fec70103e60 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7019b0c0 con 0x7fec700fee20 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec7621b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7019b280 con 0x7fec700ff7c0 2023-07-14T19:34:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec6ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 0x7fec7019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec6ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 0x7fec7019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44270/0 (socket says 172.21.15.82:44270) 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec6ffff700 1 -- 172.21.15.82:0/3888426577 learned_addr learned my addr 172.21.15.82:0/3888426577 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.082+0000 7fec6effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec700ff7c0 0x7fec701962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec700fee20 0x7fec70195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6ffff700 1 -- 172.21.15.82:0/3888426577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec700ff7c0 msgr2=0x7fec701962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6ffff700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec700ff7c0 0x7fec701962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6ffff700 1 -- 172.21.15.82:0/3888426577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec700fee20 msgr2=0x7fec70195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6ffff700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec700fee20 0x7fec70195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6ffff700 1 -- 172.21.15.82:0/3888426577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640095f0 con 0x7fec70103e60 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6ffff700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 0x7fec7019a910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec68007ee0 tx=0x7fec6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6cff9700 1 -- 172.21.15.82:0/3888426577 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68017070 con 0x7fec70103e60 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7019b560 con 0x7fec70103e60 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6cff9700 1 -- 172.21.15.82:0/3888426577 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6800f040 con 0x7fec70103e60 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec6cff9700 1 -- 172.21.15.82:0/3888426577 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68005580 con 0x7fec70103e60 2023-07-14T19:34:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.083+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701a2da0 con 0x7fec70103e60 2023-07-14T19:34:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.086+0000 7fec6cff9700 1 -- 172.21.15.82:0/3888426577 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fec6800db20 con 0x7fec70103e60 2023-07-14T19:34:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.086+0000 7fec6cff9700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fec580a12c0 0x7fec580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6cff9700 1 -- 172.21.15.82:0/3888426577 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fec680cc3a0 con 0x7fec70103e60 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fec580a12c0 0x7fec580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 Processor -- start 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 -- start start 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fec580a12c0 0x7fec580a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec60007be0 tx=0x7fec600073d0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672161550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x55967215b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559672060c50 con 0x559671fb76b0 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596720b39e0 con 0x559672161f00 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec823b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596720ace70 con 0x559672161a90 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x55967215b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40626/0 (socket says 172.21.15.82:40626) 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.087+0000 7fec6ffff700 1 -- 172.21.15.82:0/2889223193 learned_addr learned my addr 172.21.15.82:0/2889223193 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec6ffff700 1 -- 172.21.15.82:0/2889223193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 msgr2=0x55967215b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec6ffff700 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x55967215b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec6ffff700 1 -- 172.21.15.82:0/2889223193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 msgr2=0x559672161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec6ffff700 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec6ffff700 1 -- 172.21.15.82:0/2889223193 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559671edad80 con 0x559672161f00 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec6ffff700 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215bf80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec68095a00 tx=0x7fec68021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4984bc5a2cfc49e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec55ffb700 1 -- 172.21.15.82:0/2889223193 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680a1950 con 0x559672161f00 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec55ffb700 1 -- 172.21.15.82:0/2889223193 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec680978a0 con 0x559672161f00 2023-07-14T19:34:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.088+0000 7fec55ffb700 1 -- 172.21.15.82:0/2889223193 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6809d370 con 0x559672161f00 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 -- 172.21.15.82:0/2889223193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 msgr2=0x55967215bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215bf80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec68095a00 tx=0x7fec68021000 comp rx=0 tx=0).stop 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 -- 172.21.15.82:0/2889223193 shutdown_connections 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215bf80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2889223193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x55967215b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 -- 172.21.15.82:0/2889223193 >> 172.21.15.82:0/2889223193 conn(0x5596721513d0 msgr2=0x559672156530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 -- 172.21.15.82:0/2889223193 shutdown_connections 2023-07-14T19:34:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.089+0000 7fec823b3b80 1 -- 172.21.15.82:0/2889223193 wait complete. 2023-07-14T19:34:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.090+0000 7fec823b3b80 1 -- start start 2023-07-14T19:34:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.090+0000 7fec823b3b80 1 Processor -- start 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.090+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559671fb76b0 0x55967216ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.090+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559672161a90 0x55967216b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.090+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161f00 0x559672164f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec823b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967204ca60 con 0x559672161a90 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec823b3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967204c760 con 0x559671fb76b0 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec823b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559671f308c0 con 0x559672161f00 2023-07-14T19:34:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec823b3b80 10 client.0 _ll_register_callbacks cb 0x55967204a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559672161a90 0x55967216b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559671fb76b0 0x55967216ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559672161a90 0x55967216b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44282/0 (socket says 172.21.15.82:44282) 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6ffff700 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161f00 0x559672164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6effd700 1 -- 172.21.15.82:0/657232729 learned_addr learned my addr 172.21.15.82:0/657232729 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 -- 172.21.15.82:0/657232729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161f00 msgr2=0x559672164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161f00 0x559672164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 -- 172.21.15.82:0/657232729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559672161a90 msgr2=0x55967216b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559672161a90 0x55967216b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 -- 172.21.15.82:0/657232729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701a2da0 con 0x559671fb76b0 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559671fb76b0 0x55967216ac20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec60010fb0 tx=0x7fec60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a5dc9c39d89e27 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec4effd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec4effd700 1 -- 172.21.15.82:0/657232729 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60003880 con 0x559671fb76b0 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559672006490 con 0x559671fb76b0 2023-07-14T19:34:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.091+0000 7fec4effd700 1 -- 172.21.15.82:0/657232729 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fec60017070 con 0x559671fb76b0 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec4effd700 1 -- 172.21.15.82:0/657232729 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec600039e0 con 0x559671fb76b0 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 10 client.25757 fetch_fsmap learned FSMap version 40 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559671ec3c00 con 0x559671fb76b0 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec4effd700 1 -- 172.21.15.82:0/657232729 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559672006490 con 0x559671fb76b0 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec4effd700 1 -- 172.21.15.82:0/657232729 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fec60011c70 con 0x559671fb76b0 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 20 client.25757 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559672175170 0x559672177580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 4 client.25757 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55967212ad40 con 0x559672175170 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec6effd700 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559672175170 0x559672177580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec4effd700 10 client.25757 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.092+0000 7fec6effd700 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559672175170 0x559672177580 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2486c50585c76cdb server_cookie=23ca65c16a93d4cc in_seq=0 out_seq=0 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec4effd700 1 -- 172.21.15.82:0/657232729 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55967212ad40 con 0x559672175170 2023-07-14T19:34:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec4effd700 10 client.25757 handle_command_reply: tid=0 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 client.25757 shutdown 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559671fb76b0 msgr2=0x55967216ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559671fb76b0 0x55967216ac20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec60010fb0 tx=0x7fec60004750 comp rx=0 tx=0).stop 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 shutdown_connections 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559672175170 0x559672177580 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161f00 0x559672164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559671fb76b0 0x55967216ac20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 --2- 172.21.15.82:0/657232729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559672161a90 0x55967216b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec4effd700 0 client.25757 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 >> 172.21.15.82:0/657232729 conn(0x5596721513d0 msgr2=0x559672155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 shutdown_connections 2023-07-14T19:34:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 1 -- 172.21.15.82:0/657232729 wait complete. 2023-07-14T19:34:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.093+0000 7fec823b3b80 20 client.25757 trim_cache size 0 max 16384 2023-07-14T19:34:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 Processor -- start 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 -- start start 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672162b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x5596721563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559672153450 con 0x559671fb76b0 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596721618e0 con 0x559672161f00 2023-07-14T19:34:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec823b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967217c380 con 0x559672161a90 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec6f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec6f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672162b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44288/0 (socket says 172.21.15.82:44288) 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec6f7fe700 1 -- 172.21.15.82:0/2523592538 learned_addr learned my addr 172.21.15.82:0/2523592538 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec6effd700 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x5596721563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.108+0000 7fec6ffff700 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec6f7fe700 1 -- 172.21.15.82:0/2523592538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 msgr2=0x5596721563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x5596721563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec6f7fe700 1 -- 172.21.15.82:0/2523592538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 msgr2=0x55967215b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec6f7fe700 1 -- 172.21.15.82:0/2523592538 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559671f011f0 con 0x559671fb76b0 2023-07-14T19:34:57.119 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672162b90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec60010780 tx=0x7fec600107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281dd610dd2afeb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec567fc700 1 -- 172.21.15.82:0/2523592538 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6001f5f0 con 0x559671fb76b0 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec567fc700 1 -- 172.21.15.82:0/2523592538 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6001fe40 con 0x559671fb76b0 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.109+0000 7fec567fc700 1 -- 172.21.15.82:0/2523592538 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60003e60 con 0x559671fb76b0 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- 172.21.15.82:0/2523592538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 msgr2=0x559672162b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672162b90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec60010780 tx=0x7fec600107b0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- 172.21.15.82:0/2523592538 shutdown_connections 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x55967215b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x559672162b90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- 172.21.15.82:0/2523592538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x5596721563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- 172.21.15.82:0/2523592538 >> 172.21.15.82:0/2523592538 conn(0x5596721513d0 msgr2=0x559672164e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- 172.21.15.82:0/2523592538 shutdown_connections 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- 172.21.15.82:0/2523592538 wait complete. 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- start start 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 Processor -- start 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x55967216b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x559672169300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x559672169880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559672169f50 con 0x559671fb76b0 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55967216a0c0 con 0x559672161f00 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.110+0000 7fec823b3b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596721c8e20 con 0x559672161a90 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec823b3b80 10 client.0 _ll_register_callbacks cb 0x7fec60010b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x55967216b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x559672169300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x559672169300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53560/0 (socket says 172.21.15.82:53560) 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x55967216b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44294/0 (socket says 172.21.15.82:44294) 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6effd700 1 -- 172.21.15.82:0/3168041705 learned_addr learned my addr 172.21.15.82:0/3168041705 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6ffff700 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x559672169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 -- 172.21.15.82:0/3168041705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 msgr2=0x559672169300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x559672169300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 -- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 msgr2=0x559672169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:34:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x559672169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 -- 172.21.15.82:0/3168041705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559671ec3c00 con 0x559671fb76b0 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec6f7fe700 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x55967216b010 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fec60039040 tx=0x7fec6001f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cbd170231aa9a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec4effd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec4effd700 1 -- 172.21.15.82:0/3168041705 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60005050 con 0x559671fb76b0 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec4effd700 1 -- 172.21.15.82:0/3168041705 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8382+0+0 (secure 0 0 0) 0x7fec60017070 con 0x559671fb76b0 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec680978a0 con 0x559671fb76b0 2023-07-14T19:34:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec4effd700 1 -- 172.21.15.82:0/3168041705 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec600051b0 con 0x559671fb76b0 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec823b3b80 10 client.16053 fetch_fsmap learned FSMap version 40 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559671ed9250 con 0x559671fb76b0 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.111+0000 7fec4effd700 1 -- 172.21.15.82:0/3168041705 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec680978a0 con 0x559671fb76b0 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec4effd700 1 -- 172.21.15.82:0/3168041705 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fec600046e0 con 0x559671fb76b0 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec823b3b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 40 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec823b3b80 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec823b3b80 20 client.16053 populate_metadata read hostname 'smithi082' 2023-07-14T19:34:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec823b3b80 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5596721ca0f0 0x5596721cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec823b3b80 4 client.16053 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596720a5e50 con 0x5596721ca0f0 2023-07-14T19:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec6effd700 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5596721ca0f0 0x5596721cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec4effd700 10 client.16053 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.112+0000 7fec6effd700 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5596721ca0f0 0x5596721cc4d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6d33ea05221c66 server_cookie=2ba7c174767d3941 in_seq=0 out_seq=0 2023-07-14T19:34:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec4effd700 1 -- 172.21.15.82:0/3168041705 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596720a5e50 con 0x5596721ca0f0 2023-07-14T19:34:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec4effd700 10 client.16053 handle_command_reply: tid=0 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 client.16053 shutdown 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 msgr2=0x55967216b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x55967216b010 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fec60039040 tx=0x7fec6001f530 comp rx=0 tx=0).stop 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 shutdown_connections 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559672161f00 0x559672169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559671fb76b0 0x55967216b010 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5596721ca0f0 0x5596721cc4d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 --2- 172.21.15.82:0/3168041705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559672161a90 0x559672169300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 >> 172.21.15.82:0/3168041705 conn(0x5596721513d0 msgr2=0x559672163710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec4effd700 0 client.16053 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:34:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 shutdown_connections 2023-07-14T19:34:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 1 -- 172.21.15.82:0/3168041705 wait complete. 2023-07-14T19:34:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.613+0000 7fec823b3b80 20 client.16053 trim_cache size 0 max 16384 2023-07-14T19:34:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:34:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:34:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:34:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:34:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fec580a12c0 msgr2=0x7fec580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fec580a12c0 0x7fec580a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec60007be0 tx=0x7fec600073d0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 msgr2=0x7fec7019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:34:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 0x7fec7019a910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec68007ee0 tx=0x7fec6800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 shutdown_connections 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fec580a12c0 0x7fec580a3770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec70103e60 0x7fec7019a910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec700fee20 0x7fec70195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 --2- 172.21.15.82:0/3888426577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec700ff7c0 0x7fec701962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 >> 172.21.15.82:0/3888426577 conn(0x7fec70076ae0 msgr2=0x7fec70102960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 shutdown_connections 2023-07-14T19:34:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:34:57.614+0000 7fec7621b700 1 -- 172.21.15.82:0/3888426577 wait complete. 2023-07-14T19:34:57.747 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:57 smithi161 ceph-mon[129184]: pgmap v196: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 1.23k op/s 2023-07-14T19:34:57.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:57 smithi133 ceph-mon[128894]: pgmap v196: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 1.23k op/s 2023-07-14T19:34:57.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:57 smithi082 ceph-mon[127461]: pgmap v196: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 1.23k op/s 2023-07-14T19:34:59.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:34:59 smithi133 ceph-mon[128894]: pgmap v197: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 731 op/s 2023-07-14T19:34:59.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:34:59 smithi082 ceph-mon[127461]: pgmap v197: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 731 op/s 2023-07-14T19:34:59.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:34:59 smithi161 ceph-mon[129184]: pgmap v197: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 731 op/s 2023-07-14T19:35:01.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:01 smithi133 ceph-mon[128894]: pgmap v198: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.0 MiB/s wr, 718 op/s 2023-07-14T19:35:01.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:01 smithi082 ceph-mon[127461]: pgmap v198: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.0 MiB/s wr, 718 op/s 2023-07-14T19:35:01.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:01 smithi161 ceph-mon[129184]: pgmap v198: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.0 MiB/s wr, 718 op/s 2023-07-14T19:35:02.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:02 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:02.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:02 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:02.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:02 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:03.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:03 smithi133 ceph-mon[128894]: pgmap v199: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.3 MiB/s wr, 791 op/s 2023-07-14T19:35:03.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:03 smithi082 ceph-mon[127461]: pgmap v199: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.3 MiB/s wr, 791 op/s 2023-07-14T19:35:03.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:03 smithi161 ceph-mon[129184]: pgmap v199: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.3 MiB/s wr, 791 op/s 2023-07-14T19:35:05.940 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:05 smithi133 ceph-mon[128894]: pgmap v200: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.9 MiB/s wr, 727 op/s 2023-07-14T19:35:05.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:05 smithi082 ceph-mon[127461]: pgmap v200: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.9 MiB/s wr, 727 op/s 2023-07-14T19:35:05.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:05 smithi161 ceph-mon[129184]: pgmap v200: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.9 MiB/s wr, 727 op/s 2023-07-14T19:35:07.942 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:07 smithi133 ceph-mon[128894]: pgmap v201: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 626 op/s 2023-07-14T19:35:07.943 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:07 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:07.943 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:07 smithi133 ceph-mon[128894]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T19:35:07.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:07 smithi082 ceph-mon[127461]: pgmap v201: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 626 op/s 2023-07-14T19:35:07.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:07 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:07.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:07 smithi082 ceph-mon[127461]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T19:35:07.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:07 smithi161 ceph-mon[129184]: pgmap v201: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 626 op/s 2023-07-14T19:35:07.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:07 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:08.000 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:07 smithi161 ceph-mon[129184]: osdmap e87: 12 total, 12 up, 12 in 2023-07-14T19:35:09.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:09 smithi133 ceph-mon[128894]: pgmap v203: 97 pgs: 8 active+clean+snaptrim_wait, 83 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 459 op/s 2023-07-14T19:35:09.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:09 smithi082 ceph-mon[127461]: pgmap v203: 97 pgs: 8 active+clean+snaptrim_wait, 83 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 459 op/s 2023-07-14T19:35:09.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:09 smithi161 ceph-mon[129184]: pgmap v203: 97 pgs: 8 active+clean+snaptrim_wait, 83 active+clean, 6 active+clean+snaptrim; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 459 op/s 2023-07-14T19:35:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 -- 172.21.15.82:0/3277849109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8280fec20 msgr2=0x7fa8280feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 --2- 172.21.15.82:0/3277849109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8280fec20 0x7fa8280feff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa81801a9a0 tx=0x7fa81801acb0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 -- 172.21.15.82:0/3277849109 shutdown_connections 2023-07-14T19:35:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 --2- 172.21.15.82:0/3277849109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa828103fd0 0x7fa828104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 --2- 172.21.15.82:0/3277849109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8280fec20 0x7fa8280feff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 --2- 172.21.15.82:0/3277849109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8280ff530 0x7fa828103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 -- 172.21.15.82:0/3277849109 >> 172.21.15.82:0/3277849109 conn(0x7fa8280fa620 msgr2=0x7fa8280fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 -- 172.21.15.82:0/3277849109 shutdown_connections 2023-07-14T19:35:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.772+0000 7fa82e771700 1 -- 172.21.15.82:0/3277849109 wait complete. 2023-07-14T19:35:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 Processor -- start 2023-07-14T19:35:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 -- start start 2023-07-14T19:35:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 0x7fa828112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8280ff530 0x7fa828112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa828103fd0 0x7fa82810e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810f0e0 con 0x7fa8280fec20 2023-07-14T19:35:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810f250 con 0x7fa828103fd0 2023-07-14T19:35:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.773+0000 7fa82e771700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82810f410 con 0x7fa8280ff530 2023-07-14T19:35:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 0x7fa828112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa8277fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8280ff530 0x7fa828112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 0x7fa828112aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44308/0 (socket says 172.21.15.82:44308) 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 -- 172.21.15.82:0/3819445184 learned_addr learned my addr 172.21.15.82:0/3819445184 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa828103fd0 0x7fa82810e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 -- 172.21.15.82:0/3819445184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8280ff530 msgr2=0x7fa828112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8280ff530 0x7fa828112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 -- 172.21.15.82:0/3819445184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa828103fd0 msgr2=0x7fa82810e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa828103fd0 0x7fa82810e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 -- 172.21.15.82:0/3819445184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81801a600 con 0x7fa8280fec20 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa827fff700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 0x7fa828112aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa818005910 tx=0x7fa81801cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.774+0000 7fa8257fa700 1 -- 172.21.15.82:0/3819445184 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81802d070 con 0x7fa8280fec20 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.775+0000 7fa8257fa700 1 -- 172.21.15.82:0/3819445184 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa818003680 con 0x7fa8280fec20 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.775+0000 7fa8257fa700 1 -- 172.21.15.82:0/3819445184 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818031dd0 con 0x7fa8280fec20 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.775+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82810f690 con 0x7fa8280fec20 2023-07-14T19:35:10.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.775+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8281a0ae0 con 0x7fa8280fec20 2023-07-14T19:35:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.778+0000 7fa8257fa700 1 -- 172.21.15.82:0/3819445184 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa818003b70 con 0x7fa8280fec20 2023-07-14T19:35:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.779+0000 7fa8257fa700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8100a13c0 0x7fa8100a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.779+0000 7fa8257fa700 1 -- 172.21.15.82:0/3819445184 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fa8180de380 con 0x7fa8280fec20 2023-07-14T19:35:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.779+0000 7fa8277fe700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8100a13c0 0x7fa8100a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.779+0000 7fa83a909b80 1 Processor -- start 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa8277fe700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8100a13c0 0x7fa8100a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa828110340 tx=0x7fa81c009190 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 -- start start 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3384550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a337e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3283c50 con 0x5600a3384f00 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a32d69e0 con 0x5600a3384a90 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa83a909b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a32cfe70 con 0x5600a31da6b0 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa82cd0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa82cd0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44316/0 (socket says 172.21.15.82:44316) 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa82cd0e700 1 -- 172.21.15.82:0/3782971556 learned_addr learned my addr 172.21.15.82:0/3782971556 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa8277fe700 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a337e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa827fff700 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3384550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.780+0000 7fa82cd0e700 1 -- 172.21.15.82:0/3782971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 msgr2=0x5600a3384550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3384550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa82cd0e700 1 -- 172.21.15.82:0/3782971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 msgr2=0x5600a337e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a337e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa82cd0e700 1 -- 172.21.15.82:0/3782971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a30fdd80 con 0x5600a3384f00 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa827fff700 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3384550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ef80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa82000f570 tx=0x7fa82000f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed29f4a88656b79d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa80e7fc700 1 -- 172.21.15.82:0/3782971556 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820004bb0 con 0x5600a3384f00 2023-07-14T19:35:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa80e7fc700 1 -- 172.21.15.82:0/3782971556 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa820004d10 con 0x5600a3384f00 2023-07-14T19:35:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.781+0000 7fa80e7fc700 1 -- 172.21.15.82:0/3782971556 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200050c0 con 0x5600a3384f00 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 -- 172.21.15.82:0/3782971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 msgr2=0x5600a337ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ef80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa82000f570 tx=0x7fa82000f880 comp rx=0 tx=0).stop 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 -- 172.21.15.82:0/3782971556 shutdown_connections 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ef80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3384550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 --2- 172.21.15.82:0/3782971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a337e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.782+0000 7fa83a909b80 1 -- 172.21.15.82:0/3782971556 >> 172.21.15.82:0/3782971556 conn(0x5600a33743d0 msgr2=0x5600a3379530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.784+0000 7fa83a909b80 1 -- 172.21.15.82:0/3782971556 shutdown_connections 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.784+0000 7fa83a909b80 1 -- 172.21.15.82:0/3782971556 wait complete. 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 -- start start 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 Processor -- start 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a338adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 0x5600a3388000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384f00 0x5600a3388630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a326fa60 con 0x5600a3384a90 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a326f760 con 0x5600a3384f00 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a31538c0 con 0x5600a31da6b0 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.785+0000 7fa83a909b80 10 client.0 _ll_register_callbacks cb 0x5600a326d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 0x5600a3388000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 0x5600a3388000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44324/0 (socket says 172.21.15.82:44324) 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 -- 172.21.15.82:0/1916402073 learned_addr learned my addr 172.21.15.82:0/1916402073 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa827fff700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a338adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384f00 0x5600a3388630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 -- 172.21.15.82:0/1916402073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 msgr2=0x5600a338adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a338adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 -- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384f00 msgr2=0x5600a3388630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384f00 0x5600a3388630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 -- 172.21.15.82:0/1916402073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82000f1d0 con 0x5600a3384a90 2023-07-14T19:35:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8277fe700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 0x5600a3388000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0154b0 tx=0x7fa81c004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b3dd25a85030b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8077fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.786+0000 7fa8077fe700 1 -- 172.21.15.82:0/1916402073 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c015c30 con 0x5600a3384a90 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600a3229490 con 0x5600a3384a90 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa8077fe700 1 -- 172.21.15.82:0/1916402073 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fa81c015d90 con 0x5600a3384a90 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa8077fe700 1 -- 172.21.15.82:0/1916402073 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c005a50 con 0x5600a3384a90 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 10 client.16062 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5600a30e6c00 con 0x5600a3384a90 2023-07-14T19:35:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa8077fe700 1 -- 172.21.15.82:0/1916402073 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600a3229490 con 0x5600a3384a90 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa8077fe700 1 -- 172.21.15.82:0/1916402073 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa81c021440 con 0x5600a3384a90 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 10 client.16062 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 20 client.16062 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a3398310 0x5600a339a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 4 client.16062 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.787+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600a334dd40 con 0x5600a3398310 2023-07-14T19:35:10.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.789+0000 7fa827fff700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a3398310 0x5600a339a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.789+0000 7fa827fff700 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a3398310 0x5600a339a720 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da72a771d0fda4d server_cookie=f328e47436bd72bc in_seq=0 out_seq=0 2023-07-14T19:35:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.790+0000 7fa8077fe700 10 client.16062 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.790+0000 7fa8077fe700 1 -- 172.21.15.82:0/1916402073 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5600a334dd40 con 0x5600a3398310 2023-07-14T19:35:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.790+0000 7fa8077fe700 10 client.16062 handle_command_reply: tid=0 2023-07-14T19:35:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.790+0000 7fa83a909b80 1 client.16062 shutdown 2023-07-14T19:35:10.804 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 msgr2=0x5600a3388000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 0x5600a3388000 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa81c0154b0 tx=0x7fa81c004860 comp rx=0 tx=0).stop 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 shutdown_connections 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384f00 0x5600a3388630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a3398310 0x5600a339a720 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a338adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1916402073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384a90 0x5600a3388000 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 >> 172.21.15.82:0/1916402073 conn(0x5600a33743d0 msgr2=0x5600a3378c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:10.805 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa8077fe700 0 client.16062 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 shutdown_connections 2023-07-14T19:35:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 1 -- 172.21.15.82:0/1916402073 wait complete. 2023-07-14T19:35:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.791+0000 7fa83a909b80 20 client.16062 trim_cache size 0 max 16384 2023-07-14T19:35:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.815+0000 7fa83a909b80 1 Processor -- start 2023-07-14T19:35:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 -- start start 2023-07-14T19:35:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3387fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a338f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a33793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3229490 con 0x5600a3384f00 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3376470 con 0x5600a3384a90 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa83a909b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a337f7b0 con 0x5600a31da6b0 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa827fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3387fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa827fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3387fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53594/0 (socket says 172.21.15.82:53594) 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa82cd0e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a33793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa827fff700 1 -- 172.21.15.82:0/1052942922 learned_addr learned my addr 172.21.15.82:0/1052942922 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa827fff700 1 -- 172.21.15.82:0/1052942922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 msgr2=0x5600a338f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.816+0000 7fa8277fe700 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a338f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa827fff700 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a338f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa827fff700 1 -- 172.21.15.82:0/1052942922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 msgr2=0x5600a33793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa827fff700 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a33793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa827fff700 1 -- 172.21.15.82:0/1052942922 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600a31241f0 con 0x5600a31da6b0 2023-07-14T19:35:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a33793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa8277fe700 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a338f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:35:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa827fff700 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3387fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8180ab1b0 tx=0x7fa81801acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2cf0a99027bdc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa80effd700 1 -- 172.21.15.82:0/1052942922 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8180af4e0 con 0x5600a31da6b0 2023-07-14T19:35:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa80effd700 1 -- 172.21.15.82:0/1052942922 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8180b3870 con 0x5600a31da6b0 2023-07-14T19:35:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.817+0000 7fa80effd700 1 -- 172.21.15.82:0/1052942922 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8180314a0 con 0x5600a31da6b0 2023-07-14T19:35:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.818+0000 7fa83a909b80 1 -- 172.21.15.82:0/1052942922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 msgr2=0x5600a3387fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.818+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3387fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8180ab1b0 tx=0x7fa81801acb0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.819+0000 7fa83a909b80 1 -- 172.21.15.82:0/1052942922 shutdown_connections 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.819+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a33793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.819+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a31da6b0 0x5600a3387fb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.819+0000 7fa83a909b80 1 --2- 172.21.15.82:0/1052942922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a3384a90 0x5600a338f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.819+0000 7fa83a909b80 1 -- 172.21.15.82:0/1052942922 >> 172.21.15.82:0/1052942922 conn(0x5600a33743d0 msgr2=0x5600a33878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.820+0000 7fa83a909b80 1 -- 172.21.15.82:0/1052942922 shutdown_connections 2023-07-14T19:35:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.820+0000 7fa83a909b80 1 -- 172.21.15.82:0/1052942922 wait complete. 2023-07-14T19:35:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.820+0000 7fa83a909b80 1 -- start start 2023-07-14T19:35:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.820+0000 7fa83a909b80 1 Processor -- start 2023-07-14T19:35:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 0x5600a3381130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a3384a90 0x5600a3381670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3380510 con 0x5600a3384f00 2023-07-14T19:35:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3380680 con 0x5600a31da6b0 2023-07-14T19:35:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600a3380840 con 0x5600a3384a90 2023-07-14T19:35:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa83a909b80 10 client.0 _ll_register_callbacks cb 0x7fa81c00ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa827fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 0x5600a3381130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa827fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 0x5600a3381130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40682/0 (socket says 172.21.15.82:40682) 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa827fff700 1 -- 172.21.15.82:0/638141540 learned_addr learned my addr 172.21.15.82:0/638141540 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.821+0000 7fa82cd0e700 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa827fff700 1 -- 172.21.15.82:0/638141540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a3384a90 msgr2=0x5600a3381670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa827fff700 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a3384a90 0x5600a3381670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa827fff700 1 -- 172.21.15.82:0/638141540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 msgr2=0x5600a337ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa827fff700 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa827fff700 1 -- 172.21.15.82:0/638141540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600a30fdd80 con 0x5600a31da6b0 2023-07-14T19:35:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa827fff700 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 0x5600a3381130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa8180a7f20 tx=0x7fa8180ab600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ed24fe287b5185 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa8077fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:35:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa8077fe700 1 -- 172.21.15.82:0/638141540 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8180a95f0 con 0x5600a31da6b0 2023-07-14T19:35:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.822+0000 7fa8077fe700 1 -- 172.21.15.82:0/638141540 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fa818031410 con 0x5600a31da6b0 2023-07-14T19:35:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.823+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa820004d10 con 0x5600a31da6b0 2023-07-14T19:35:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.823+0000 7fa8077fe700 1 -- 172.21.15.82:0/638141540 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8180a9850 con 0x5600a31da6b0 2023-07-14T19:35:10.834 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.823+0000 7fa83a909b80 10 client.25778 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.823+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fa8281a0ae0 con 0x5600a31da6b0 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa8077fe700 1 -- 172.21.15.82:0/638141540 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa820004d10 con 0x5600a31da6b0 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa8077fe700 1 -- 172.21.15.82:0/638141540 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa8180aa9a0 con 0x5600a31da6b0 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa83a909b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa83a909b80 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa83a909b80 20 client.25778 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa83a909b80 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a33eca30 0x5600a33eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa83a909b80 4 client.25778 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:35:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5600a32c8e50 con 0x5600a33eca30 2023-07-14T19:35:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.824+0000 7fa8277fe700 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a33eca30 0x5600a33eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.825+0000 7fa8277fe700 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a33eca30 0x5600a33eee40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2df304fea584b server_cookie=e21d3b91d1f266a1 in_seq=0 out_seq=0 2023-07-14T19:35:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:10.825+0000 7fa8077fe700 10 client.25778 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.258+0000 7fa8077fe700 1 -- 172.21.15.82:0/638141540 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5600a32c8e50 con 0x5600a33eca30 2023-07-14T19:35:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.258+0000 7fa8077fe700 10 client.25778 handle_command_reply: tid=0 2023-07-14T19:35:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.258+0000 7fa83a909b80 1 client.25778 shutdown 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 msgr2=0x5600a3381130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 0x5600a3381130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa8180a7f20 tx=0x7fa8180ab600 comp rx=0 tx=0).stop 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 shutdown_connections 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5600a3384f00 0x5600a337ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5600a31da6b0 0x5600a3381130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5600a33eca30 0x5600a33eee40 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 --2- 172.21.15.82:0/638141540 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5600a3384a90 0x5600a3381670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 >> 172.21.15.82:0/638141540 conn(0x5600a33743d0 msgr2=0x5600a3385300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa8077fe700 0 client.25778 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 shutdown_connections 2023-07-14T19:35:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 1 -- 172.21.15.82:0/638141540 wait complete. 2023-07-14T19:35:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.259+0000 7fa83a909b80 20 client.25778 trim_cache size 0 max 16384 2023-07-14T19:35:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:35:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:35:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:35:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:35:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8100a13c0 msgr2=0x7fa8100a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8100a13c0 0x7fa8100a3870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa828110340 tx=0x7fa81c009190 comp rx=0 tx=0).stop 2023-07-14T19:35:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 msgr2=0x7fa828112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 0x7fa828112aa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa818005910 tx=0x7fa81801cf10 comp rx=0 tx=0).stop 2023-07-14T19:35:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 shutdown_connections 2023-07-14T19:35:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa8100a13c0 0x7fa8100a3870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa828103fd0 0x7fa82810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8280fec20 0x7fa828112aa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 --2- 172.21.15.82:0/3819445184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8280ff530 0x7fa828112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 >> 172.21.15.82:0/3819445184 conn(0x7fa8280fa620 msgr2=0x7fa8281019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 shutdown_connections 2023-07-14T19:35:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:11.260+0000 7fa82e771700 1 -- 172.21.15.82:0/3819445184 wait complete. 2023-07-14T19:35:11.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:11 smithi133 ceph-mon[128894]: pgmap v204: 97 pgs: 27 active+clean+snaptrim_wait, 55 active+clean, 15 active+clean+snaptrim; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2023-07-14T19:35:11.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:11 smithi082 ceph-mon[127461]: pgmap v204: 97 pgs: 27 active+clean+snaptrim_wait, 55 active+clean, 15 active+clean+snaptrim; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2023-07-14T19:35:11.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:11 smithi161 ceph-mon[129184]: pgmap v204: 97 pgs: 27 active+clean+snaptrim_wait, 55 active+clean, 15 active+clean+snaptrim; 1.6 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 332 op/s 2023-07-14T19:35:13.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:13 smithi133 ceph-mon[128894]: pgmap v205: 97 pgs: 21 active+clean+snaptrim_wait, 65 active+clean, 11 active+clean+snaptrim; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2023-07-14T19:35:13.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:13 smithi082 ceph-mon[127461]: pgmap v205: 97 pgs: 21 active+clean+snaptrim_wait, 65 active+clean, 11 active+clean+snaptrim; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2023-07-14T19:35:13.998 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:13 smithi161 ceph-mon[129184]: pgmap v205: 97 pgs: 21 active+clean+snaptrim_wait, 65 active+clean, 11 active+clean+snaptrim; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2023-07-14T19:35:14.946 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:14 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:35:14.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:14 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:35:14.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:14 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:35:15.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:15 smithi133 ceph-mon[128894]: pgmap v206: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-14T19:35:15.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:15 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:35:15.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:15 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:35:15.942 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:15 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:35:15.971 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:15 smithi082 ceph-mon[127461]: pgmap v206: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-14T19:35:15.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:15 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:35:15.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:15 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:35:15.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:15 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:35:15.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:15 smithi161 ceph-mon[129184]: pgmap v206: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2023-07-14T19:35:15.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:15 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:35:15.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:15 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:35:15.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:15 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:35:17.972 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:17 smithi082 ceph-mon[127461]: pgmap v207: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2023-07-14T19:35:17.998 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:17 smithi133 ceph-mon[128894]: pgmap v207: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2023-07-14T19:35:17.999 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:17 smithi161 ceph-mon[129184]: pgmap v207: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2023-07-14T19:35:19.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:18 smithi082 ceph-mon[127461]: pgmap v208: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 255 op/s 2023-07-14T19:35:19.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:18 smithi082 ceph-mon[127461]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T19:35:19.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:18 smithi161 ceph-mon[129184]: pgmap v208: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 255 op/s 2023-07-14T19:35:19.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:18 smithi161 ceph-mon[129184]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T19:35:19.714 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:18 smithi133 ceph-mon[128894]: pgmap v208: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 255 op/s 2023-07-14T19:35:19.715 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:18 smithi133 ceph-mon[128894]: osdmap e88: 12 total, 12 up, 12 in 2023-07-14T19:35:21.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:20 smithi082 ceph-mon[127461]: pgmap v210: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 295 op/s 2023-07-14T19:35:21.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:20 smithi161 ceph-mon[129184]: pgmap v210: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 295 op/s 2023-07-14T19:35:21.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:20 smithi133 ceph-mon[128894]: pgmap v210: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 295 op/s 2023-07-14T19:35:23.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:22 smithi082 ceph-mon[127461]: pgmap v211: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 295 op/s 2023-07-14T19:35:23.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:22 smithi161 ceph-mon[129184]: pgmap v211: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 295 op/s 2023-07-14T19:35:23.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:22 smithi133 ceph-mon[128894]: pgmap v211: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 295 op/s 2023-07-14T19:35:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 -- 172.21.15.82:0/3043846427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4104ef0 msgr2=0x7fe8a410cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 --2- 172.21.15.82:0/3043846427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4104ef0 0x7fe8a410cac0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe8940099c0 tx=0x7fe894009cd0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 -- 172.21.15.82:0/3043846427 shutdown_connections 2023-07-14T19:35:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 --2- 172.21.15.82:0/3043846427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4104ef0 0x7fe8a410cac0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 --2- 172.21.15.82:0/3043846427 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 --2- 172.21.15.82:0/3043846427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4100850 0x7fe8a4104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 -- 172.21.15.82:0/3043846427 >> 172.21.15.82:0/3043846427 conn(0x7fe8a40fb720 msgr2=0x7fe8a40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 -- 172.21.15.82:0/3043846427 shutdown_connections 2023-07-14T19:35:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.411+0000 7fe8ab688700 1 -- 172.21.15.82:0/3043846427 wait complete. 2023-07-14T19:35:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 Processor -- start 2023-07-14T19:35:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 -- start start 2023-07-14T19:35:24.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 0x7fe8a419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4104ef0 0x7fe8a4195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a4196130 con 0x7fe8a4100850 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a41962a0 con 0x7fe8a4104ef0 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8ab688700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a4196460 con 0x7fe8a40ffeb0 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 0x7fe8a419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53612/0 (socket says 172.21.15.82:53612) 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 0x7fe8a419c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44350/0 (socket says 172.21.15.82:44350) 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a9424700 1 -- 172.21.15.82:0/1924786229 learned_addr learned my addr 172.21.15.82:0/1924786229 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4104ef0 0x7fe8a4195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1924786229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 msgr2=0x7fe8a419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a419bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1924786229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4104ef0 msgr2=0x7fe8a4195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.412+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4104ef0 0x7fe8a4195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1924786229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8940095f0 con 0x7fe8a4100850 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4104ef0 0x7fe8a4195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 0x7fe8a419c1a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4100c20 tx=0x7fe89800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a27fc700 1 -- 172.21.15.82:0/1924786229 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898010900 con 0x7fe8a4100850 2023-07-14T19:35:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a4196740 con 0x7fe8a4100850 2023-07-14T19:35:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a27fc700 1 -- 172.21.15.82:0/1924786229 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89800c430 con 0x7fe8a4100850 2023-07-14T19:35:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8a27fc700 1 -- 172.21.15.82:0/1924786229 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898018680 con 0x7fe8a4100850 2023-07-14T19:35:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.413+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a41a2c50 con 0x7fe8a4100850 2023-07-14T19:35:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.415+0000 7fe8a27fc700 1 -- 172.21.15.82:0/1924786229 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe89800c5a0 con 0x7fe8a4100850 2023-07-14T19:35:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.415+0000 7fe8a27fc700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe88c0a1340 0x7fe88c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.415+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe88c0a1340 0x7fe88c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.416+0000 7fe8a27fc700 1 -- 172.21.15.82:0/1924786229 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fe898014070 con 0x7fe8a4100850 2023-07-14T19:35:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.416+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe88c0a1340 0x7fe88c0a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009c10 tx=0x7fe89c00b040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.416+0000 7fe8b7820b80 1 Processor -- start 2023-07-14T19:35:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.416+0000 7fe8b7820b80 1 -- start start 2023-07-14T19:35:24.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c417b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c4175f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c407ac50 con 0x5594c417ba90 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c40cd9e0 con 0x5594c3fd16b0 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c40c6e70 con 0x5594c417bf00 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44358/0 (socket says 172.21.15.82:44358) 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c417b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1029761891 learned_addr learned my addr 172.21.15.82:0/1029761891 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c4175f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1029761891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 msgr2=0x5594c4175f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c4175f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1029761891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 msgr2=0x5594c417b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c417b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.434 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 -- 172.21.15.82:0/1029761891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594c3ef4d80 con 0x5594c417ba90 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c417b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41759e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe898009230 tx=0x7fe898099900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6614e63c366b20 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe88b7fe700 1 -- 172.21.15.82:0/1029761891 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980a22c0 con 0x5594c417ba90 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe88b7fe700 1 -- 172.21.15.82:0/1029761891 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89800c880 con 0x5594c417ba90 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.417+0000 7fe88b7fe700 1 -- 172.21.15.82:0/1029761891 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980995c0 con 0x5594c417ba90 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1029761891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 msgr2=0x5594c41759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41759e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe898009230 tx=0x7fe898099900 comp rx=0 tx=0).stop 2023-07-14T19:35:24.435 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1029761891 shutdown_connections 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c4175f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c417b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1029761891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41759e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1029761891 >> 172.21.15.82:0/1029761891 conn(0x5594c416b3d0 msgr2=0x5594c4170530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1029761891 shutdown_connections 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1029761891 wait complete. 2023-07-14T19:35:24.436 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 -- start start 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.418+0000 7fe8b7820b80 1 Processor -- start 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c4184e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417ba90 0x5594c4185350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 0x5594c417f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c4066a60 con 0x5594c417bf00 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c4066760 con 0x5594c3fd16b0 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c3f4a8c0 con 0x5594c417ba90 2023-07-14T19:35:24.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8b7820b80 10 client.0 _ll_register_callbacks cb 0x5594c4064700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c4184e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 0x5594c417f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c4184e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40710/0 (socket says 172.21.15.82:40710) 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9424700 1 -- 172.21.15.82:0/1883704948 learned_addr learned my addr 172.21.15.82:0/1883704948 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 0x5594c417f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44366/0 (socket says 172.21.15.82:44366) 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417ba90 0x5594c4185350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 -- 172.21.15.82:0/1883704948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417ba90 msgr2=0x5594c4185350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417ba90 0x5594c4185350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 -- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 msgr2=0x5594c4184e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c4184e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9c25700 1 -- 172.21.15.82:0/1883704948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c00e5a0 con 0x5594c417bf00 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.419+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c4184e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 0x5594c417f0a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe89400b580 tx=0x7fe89400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcbc8a578961933 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe888ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:35:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe888ff9700 1 -- 172.21.15.82:0/1883704948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8940054e0 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594c4020490 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe888ff9700 1 -- 172.21.15.82:0/1883704948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fe894005640 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe888ff9700 1 -- 172.21.15.82:0/1883704948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89400f800 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 10 client.16080 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fe8a41a2c50 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe888ff9700 1 -- 172.21.15.82:0/1883704948 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe894031020 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe888ff9700 1 -- 172.21.15.82:0/1883704948 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe89402b070 con 0x5594c417bf00 2023-07-14T19:35:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 10 client.16080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 20 client.16080 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c418f310 0x5594c4191720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 4 client.16080 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594c4144d40 con 0x5594c418f310 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.420+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c418f310 0x5594c4191720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe8a9424700 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c418f310 0x5594c4191720 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d957f7aad72d87dc server_cookie=b29c5ace23ac6df2 in_seq=0 out_seq=0 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe888ff9700 10 client.16080 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe888ff9700 1 -- 172.21.15.82:0/1883704948 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5594c4144d40 con 0x5594c418f310 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe888ff9700 10 client.16080 handle_command_reply: tid=0 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe8b7820b80 1 client.16080 shutdown 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 msgr2=0x5594c417f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 0x5594c417f0a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe89400b580 tx=0x7fe89400f690 comp rx=0 tx=0).stop 2023-07-14T19:35:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.421+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 shutdown_connections 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417bf00 0x5594c417f0a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c3fd16b0 0x5594c4184e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c418f310 0x5594c4191720 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/1883704948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417ba90 0x5594c4185350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe888ff9700 0 client.16080 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 >> 172.21.15.82:0/1883704948 conn(0x5594c416b3d0 msgr2=0x5594c416fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 shutdown_connections 2023-07-14T19:35:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 1 -- 172.21.15.82:0/1883704948 wait complete. 2023-07-14T19:35:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.422+0000 7fe8b7820b80 20 client.16080 trim_cache size 0 max 16384 2023-07-14T19:35:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 Processor -- start 2023-07-14T19:35:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 -- start start 2023-07-14T19:35:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c3fd16b0 0x5594c417a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417bf00 0x5594c4177fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c4171660 con 0x5594c417ba90 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c4172480 con 0x5594c417bf00 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c4179d60 con 0x5594c3fd16b0 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41703b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44372/0 (socket says 172.21.15.82:44372) 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 -- 172.21.15.82:0/3383239812 learned_addr learned my addr 172.21.15.82:0/3383239812 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a9424700 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c3fd16b0 0x5594c417a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417bf00 0x5594c4177fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 -- 172.21.15.82:0/3383239812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c3fd16b0 msgr2=0x5594c417a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c3fd16b0 0x5594c417a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 -- 172.21.15.82:0/3383239812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417bf00 msgr2=0x5594c4177fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417bf00 0x5594c4177fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 -- 172.21.15.82:0/3383239812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594c3eddc00 con 0x5594c417ba90 2023-07-14T19:35:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417bf00 0x5594c4177fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:35:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.437+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41703b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe8980968e0 tx=0x7fe8980092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e380381adfc982a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe88bfff700 1 -- 172.21.15.82:0/3383239812 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89800c9f0 con 0x5594c417ba90 2023-07-14T19:35:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe88bfff700 1 -- 172.21.15.82:0/3383239812 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8980a22c0 con 0x5594c417ba90 2023-07-14T19:35:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe88bfff700 1 -- 172.21.15.82:0/3383239812 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980a2090 con 0x5594c417ba90 2023-07-14T19:35:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3383239812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 msgr2=0x5594c41703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41703b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe8980968e0 tx=0x7fe8980092d0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3383239812 shutdown_connections 2023-07-14T19:35:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417bf00 0x5594c4177fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c3fd16b0 0x5594c417a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3383239812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c417ba90 0x5594c41703b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3383239812 >> 172.21.15.82:0/3383239812 conn(0x5594c416b3d0 msgr2=0x5594c417a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3383239812 shutdown_connections 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3383239812 wait complete. 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.438+0000 7fe8b7820b80 1 -- start start 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 Processor -- start 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 0x5594c4182a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417ba90 0x5594c4182f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c41e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c4183740 con 0x5594c3fd16b0 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c41838b0 con 0x5594c417ba90 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594c41e6930 con 0x5594c417bf00 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8b7820b80 10 client.0 _ll_register_callbacks cb 0x7fe89400be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 0x5594c4182a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 0x5594c4182a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44376/0 (socket says 172.21.15.82:44376) 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 -- 172.21.15.82:0/3502221857 learned_addr learned my addr 172.21.15.82:0/3502221857 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a8c23700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417ba90 0x5594c4182f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9c25700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c41e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 -- 172.21.15.82:0/3502221857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 msgr2=0x5594c41e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c41e63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 -- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417ba90 msgr2=0x5594c4182f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417ba90 0x5594c4182f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a9424700 1 -- 172.21.15.82:0/3502221857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594c3ef4d80 con 0x5594c3fd16b0 2023-07-14T19:35:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.439+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417ba90 0x5594c4182f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8a9424700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 0x5594c4182a50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5594c4192000 tx=0x7fe89c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c59b8a55b72b78 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe888ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe888ff9700 1 -- 172.21.15.82:0/3502221857 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c02b070 con 0x5594c3fd16b0 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe888ff9700 1 -- 172.21.15.82:0/3502221857 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fe89c02b3d0 con 0x5594c3fd16b0 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe89800c880 con 0x5594c3fd16b0 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe888ff9700 1 -- 172.21.15.82:0/3502221857 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c015900 con 0x5594c3fd16b0 2023-07-14T19:35:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 10 client.16086 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5594c3f1b1f0 con 0x5594c3fd16b0 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe888ff9700 1 -- 172.21.15.82:0/3502221857 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89800c880 con 0x5594c3fd16b0 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe888ff9700 1 -- 172.21.15.82:0/3502221857 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe89c03b020 con 0x5594c3fd16b0 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 20 client.16086 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c41e80e0 0x5594c41ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:24.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 4 client.16086 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:35:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5594c40bfe50 con 0x5594c41e80e0 2023-07-14T19:35:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.440+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c41e80e0 0x5594c41ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.441+0000 7fe8a8c23700 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c41e80e0 0x5594c41ea4f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee87541cb32cac8b server_cookie=e8ce32d94fa37ea9 in_seq=0 out_seq=0 2023-07-14T19:35:24.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.441+0000 7fe888ff9700 10 client.16086 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe888ff9700 1 -- 172.21.15.82:0/3502221857 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5594c40bfe50 con 0x5594c41e80e0 2023-07-14T19:35:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe888ff9700 10 client.16086 handle_command_reply: tid=0 2023-07-14T19:35:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 client.16086 shutdown 2023-07-14T19:35:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 msgr2=0x5594c4182a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 0x5594c4182a50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5594c4192000 tx=0x7fe89c004ad0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 shutdown_connections 2023-07-14T19:35:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5594c41e80e0 0x5594c41ea4f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5594c417bf00 0x5594c41e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe888ff9700 0 client.16086 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594c3fd16b0 0x5594c4182a50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 --2- 172.21.15.82:0/3502221857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594c417ba90 0x5594c4182f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.916+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 >> 172.21.15.82:0/3502221857 conn(0x5594c416b3d0 msgr2=0x5594c41749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 shutdown_connections 2023-07-14T19:35:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8b7820b80 1 -- 172.21.15.82:0/3502221857 wait complete. 2023-07-14T19:35:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8b7820b80 20 client.16086 trim_cache size 0 max 16384 2023-07-14T19:35:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:35:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:35:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:35:24.928 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:35:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe88c0a1340 msgr2=0x7fe88c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe88c0a1340 0x7fe88c0a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009c10 tx=0x7fe89c00b040 comp rx=0 tx=0).stop 2023-07-14T19:35:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 msgr2=0x7fe8a419c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 0x7fe8a419c1a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4100c20 tx=0x7fe89800eca0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 shutdown_connections 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fe88c0a1340 0x7fe88c0a37f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.917+0000 7fe8ab688700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8a4104ef0 0x7fe8a4195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.918+0000 7fe8ab688700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8a40ffeb0 0x7fe8a419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.918+0000 7fe8ab688700 1 --2- 172.21.15.82:0/1924786229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8a4100850 0x7fe8a419c1a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.918+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 >> 172.21.15.82:0/1924786229 conn(0x7fe8a40fb720 msgr2=0x7fe8a40fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.918+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 shutdown_connections 2023-07-14T19:35:24.931 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:24.918+0000 7fe8ab688700 1 -- 172.21.15.82:0/1924786229 wait complete. 2023-07-14T19:35:25.224 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:24 smithi082 ceph-mon[127461]: pgmap v212: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-07-14T19:35:25.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:24 smithi161 ceph-mon[129184]: pgmap v212: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-07-14T19:35:25.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:24 smithi133 ceph-mon[128894]: pgmap v212: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 279 op/s 2023-07-14T19:35:27.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:26 smithi082 ceph-mon[127461]: pgmap v213: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2023-07-14T19:35:27.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:26 smithi161 ceph-mon[129184]: pgmap v213: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2023-07-14T19:35:27.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:26 smithi133 ceph-mon[128894]: pgmap v213: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 298 op/s 2023-07-14T19:35:29.222 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:28 smithi082 ceph-mon[127461]: pgmap v214: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 337 op/s 2023-07-14T19:35:29.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:28 smithi161 ceph-mon[129184]: pgmap v214: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 337 op/s 2023-07-14T19:35:29.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:28 smithi133 ceph-mon[128894]: pgmap v214: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 337 op/s 2023-07-14T19:35:31.249 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:30 smithi161 ceph-mon[129184]: pgmap v215: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2023-07-14T19:35:31.441 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:30 smithi133 ceph-mon[128894]: pgmap v215: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2023-07-14T19:35:31.472 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:30 smithi082 ceph-mon[127461]: pgmap v215: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 350 op/s 2023-07-14T19:35:33.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:32 smithi161 ceph-mon[129184]: pgmap v216: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 335 op/s 2023-07-14T19:35:33.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:32 smithi133 ceph-mon[128894]: pgmap v216: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 335 op/s 2023-07-14T19:35:33.472 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:32 smithi082 ceph-mon[127461]: pgmap v216: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 335 op/s 2023-07-14T19:35:35.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:34 smithi161 ceph-mon[129184]: pgmap v217: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 330 op/s 2023-07-14T19:35:35.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:34 smithi133 ceph-mon[128894]: pgmap v217: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 330 op/s 2023-07-14T19:35:35.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:34 smithi082 ceph-mon[127461]: pgmap v217: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 330 op/s 2023-07-14T19:35:37.250 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:36 smithi161 ceph-mon[129184]: pgmap v218: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 371 op/s 2023-07-14T19:35:37.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:36 smithi133 ceph-mon[128894]: pgmap v218: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 371 op/s 2023-07-14T19:35:37.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:36 smithi082 ceph-mon[127461]: pgmap v218: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 371 op/s 2023-07-14T19:35:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 -- 172.21.15.82:0/2308109822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 msgr2=0x7ff5c010b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2308109822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c010b9d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff5b800b0d0 tx=0x7ff5b800b3e0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 -- 172.21.15.82:0/2308109822 shutdown_connections 2023-07-14T19:35:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2308109822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c010b9d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2308109822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c00ffa10 0x7ff5c00ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2308109822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c01003b0 0x7ff5c0104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 -- 172.21.15.82:0/2308109822 >> 172.21.15.82:0/2308109822 conn(0x7ff5c0074cc0 msgr2=0x7ff5c00750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 -- 172.21.15.82:0/2308109822 shutdown_connections 2023-07-14T19:35:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.031+0000 7ff5c839a700 1 -- 172.21.15.82:0/2308109822 wait complete. 2023-07-14T19:35:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 Processor -- start 2023-07-14T19:35:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 -- start start 2023-07-14T19:35:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c00ffa10 0x7ff5c010c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c01003b0 0x7ff5c010c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c01a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01a45a0 con 0x7ff5c0104fe0 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01a46e0 con 0x7ff5c01003b0 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c839a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01a4850 con 0x7ff5c00ffa10 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c01a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c01a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44394/0 (socket says 172.21.15.82:44394) 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c6937700 1 -- 172.21.15.82:0/2582607110 learned_addr learned my addr 172.21.15.82:0/2582607110 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c5935700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c01003b0 0x7ff5c010c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.032+0000 7ff5c6136700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c00ffa10 0x7ff5c010c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c6937700 1 -- 172.21.15.82:0/2582607110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c00ffa10 msgr2=0x7ff5c010c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c6937700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c00ffa10 0x7ff5c010c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c6937700 1 -- 172.21.15.82:0/2582607110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c01003b0 msgr2=0x7ff5c010c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c6937700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c01003b0 0x7ff5c010c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c6937700 1 -- 172.21.15.82:0/2582607110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8009d00 con 0x7ff5c0104fe0 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c5935700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c01003b0 0x7ff5c010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:35:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c6937700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c01a4060 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8007870 tx=0x7ff5b80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5b77fe700 1 -- 172.21.15.82:0/2582607110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800e040 con 0x7ff5c0104fe0 2023-07-14T19:35:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c01a4ad0 con 0x7ff5c0104fe0 2023-07-14T19:35:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5b77fe700 1 -- 172.21.15.82:0/2582607110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b8005070 con 0x7ff5c0104fe0 2023-07-14T19:35:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5b77fe700 1 -- 172.21.15.82:0/2582607110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b801b6a0 con 0x7ff5c0104fe0 2023-07-14T19:35:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.033+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c01a4f50 con 0x7ff5c0104fe0 2023-07-14T19:35:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.035+0000 7ff5b77fe700 1 -- 172.21.15.82:0/2582607110 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff5b80051e0 con 0x7ff5c0104fe0 2023-07-14T19:35:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.035+0000 7ff5b77fe700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5ac0a12c0 0x7ff5ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.036+0000 7ff5c6136700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5ac0a12c0 0x7ff5ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.036+0000 7ff5b77fe700 1 -- 172.21.15.82:0/2582607110 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7ff5b80cde70 con 0x7ff5c0104fe0 2023-07-14T19:35:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.036+0000 7ff5c6136700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5ac0a12c0 0x7ff5ac0a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005950 tx=0x7ff5b000b410 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.036+0000 7ff5d4532b80 1 Processor -- start 2023-07-14T19:35:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 -- start start 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0a90 0x559916b9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916a9fc50 con 0x559916ba0f00 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916af29e0 con 0x559916ba0a90 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916aebe70 con 0x5599169f66b0 2023-07-14T19:35:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44402/0 (socket says 172.21.15.82:44402) 2023-07-14T19:35:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 -- 172.21.15.82:0/1530872689 learned_addr learned my addr 172.21.15.82:0/1530872689 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6136700 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c5935700 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0a90 0x559916b9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 -- 172.21.15.82:0/1530872689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 msgr2=0x559916ba0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 -- 172.21.15.82:0/1530872689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0a90 msgr2=0x559916b9a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0a90 0x559916b9a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.037+0000 7ff5c6937700 1 -- 172.21.15.82:0/1530872689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559916919d80 con 0x559916ba0f00 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5c6937700 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9af80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff5b809afc0 tx=0x7ff5b809ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cf19fc9c2d88d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5b4daa700 1 -- 172.21.15.82:0/1530872689 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b809b030 con 0x559916ba0f00 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5b4daa700 1 -- 172.21.15.82:0/1530872689 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b809f060 con 0x559916ba0f00 2023-07-14T19:35:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5b4daa700 1 -- 172.21.15.82:0/1530872689 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b809a520 con 0x559916ba0f00 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 -- 172.21.15.82:0/1530872689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 msgr2=0x559916b9af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9af80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff5b809afc0 tx=0x7ff5b809ab70 comp rx=0 tx=0).stop 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 -- 172.21.15.82:0/1530872689 shutdown_connections 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9af80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/1530872689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0a90 0x559916b9a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.038+0000 7ff5d4532b80 1 -- 172.21.15.82:0/1530872689 >> 172.21.15.82:0/1530872689 conn(0x559916b903d0 msgr2=0x559916b95530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 -- 172.21.15.82:0/1530872689 shutdown_connections 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 -- 172.21.15.82:0/1530872689 wait complete. 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 -- start start 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 Processor -- start 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 0x559916baa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0f00 0x559916ba3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916a8ba60 con 0x559916ba0a90 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916a8b760 con 0x559916ba0f00 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55991696f8c0 con 0x5599169f66b0 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5d4532b80 10 client.0 _ll_register_callbacks cb 0x559916a89700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5c5935700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 0x559916baa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5c5935700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 0x559916baa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44406/0 (socket says 172.21.15.82:44406) 2023-07-14T19:35:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.039+0000 7ff5c5935700 1 -- 172.21.15.82:0/4080672603 learned_addr learned my addr 172.21.15.82:0/4080672603 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c6136700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c6937700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0f00 0x559916ba3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c5935700 1 -- 172.21.15.82:0/4080672603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 msgr2=0x559916ba9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c5935700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c5935700 1 -- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0f00 msgr2=0x559916ba3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c5935700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0f00 0x559916ba3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c5935700 1 -- 172.21.15.82:0/4080672603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c01a4f50 con 0x559916ba0a90 2023-07-14T19:35:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5c5935700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 0x559916baa060 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009d20 tx=0x7ff5bc003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c71a88a621c038d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/4080672603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc016070 con 0x559916ba0a90 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/4080672603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7ff5bc011440 con 0x559916ba0a90 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559916a45490 con 0x559916ba0a90 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/4080672603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5bc011600 con 0x559916ba0a90 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5d4532b80 10 client.16098 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559916902c00 con 0x559916ba0a90 2023-07-14T19:35:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.040+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/4080672603 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559916a45490 con 0x559916ba0a90 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/4080672603 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff5bc022070 con 0x559916ba0a90 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5d4532b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5d4532b80 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5d4532b80 20 client.16098 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916bb4150 0x559916bb6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5d4532b80 4 client.16098 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:35:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559916b69d40 con 0x559916bb4150 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5c6136700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916bb4150 0x559916bb6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5a9ffb700 10 client.16098 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.041+0000 7ff5c6136700 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916bb4150 0x559916bb6560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2348d26576b76955 server_cookie=ddecd3e856e8aed6 in_seq=0 out_seq=0 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/4080672603 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559916b69d40 con 0x559916bb4150 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5a9ffb700 10 client.16098 handle_command_reply: tid=0 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 client.16098 shutdown 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 msgr2=0x559916baa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 0x559916baa060 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009d20 tx=0x7ff5bc003960 comp rx=0 tx=0).stop 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 shutdown_connections 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559916ba0f00 0x559916ba3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916bb4150 0x559916bb6560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5599169f66b0 0x559916ba9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4080672603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0a90 0x559916baa060 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5a9ffb700 0 client.16098 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 >> 172.21.15.82:0/4080672603 conn(0x559916b903d0 msgr2=0x559916b94c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 shutdown_connections 2023-07-14T19:35:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4080672603 wait complete. 2023-07-14T19:35:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.042+0000 7ff5d4532b80 20 client.16098 trim_cache size 0 max 16384 2023-07-14T19:35:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 Processor -- start 2023-07-14T19:35:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 -- start start 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916b953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916b96d70 con 0x559916ba0f00 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916b92450 con 0x5599169f66b0 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916bbb360 con 0x559916ba0a90 2023-07-14T19:35:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44416/0 (socket says 172.21.15.82:44416) 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6136700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 -- 172.21.15.82:0/4117622383 learned_addr learned my addr 172.21.15.82:0/4117622383 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c5935700 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916b953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 -- 172.21.15.82:0/4117622383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 msgr2=0x559916b953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916b953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.070 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 -- 172.21.15.82:0/4117622383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 msgr2=0x559916ba1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.057+0000 7ff5c6937700 1 -- 172.21.15.82:0/4117622383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599169401f0 con 0x559916ba0f00 2023-07-14T19:35:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5c6937700 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9a9e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80079c0 tx=0x7ff5b80974a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6a714fccc27c11 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5b55ab700 1 -- 172.21.15.82:0/4117622383 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b809b030 con 0x559916ba0f00 2023-07-14T19:35:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5b55ab700 1 -- 172.21.15.82:0/4117622383 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b80a33f0 con 0x559916ba0f00 2023-07-14T19:35:38.071 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5b55ab700 1 -- 172.21.15.82:0/4117622383 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80a2f60 con 0x559916ba0f00 2023-07-14T19:35:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4117622383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 msgr2=0x559916b9a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9a9e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80079c0 tx=0x7ff5b80974a0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4117622383 shutdown_connections 2023-07-14T19:35:38.074 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916b9a9e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/4117622383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916b953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4117622383 >> 172.21.15.82:0/4117622383 conn(0x559916b903d0 msgr2=0x559916ba1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4117622383 shutdown_connections 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.058+0000 7ff5d4532b80 1 -- 172.21.15.82:0/4117622383 wait complete. 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 -- start start 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 Processor -- start 2023-07-14T19:35:38.075 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916ba7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916ba8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916c06ed0 con 0x559916ba0f00 2023-07-14T19:35:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916c07040 con 0x5599169f66b0 2023-07-14T19:35:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559916c07200 con 0x559916ba0a90 2023-07-14T19:35:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5d4532b80 10 client.0 _ll_register_callbacks cb 0x7ff5bc0097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916ba8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.077 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40766/0 (socket says 172.21.15.82:40766) 2023-07-14T19:35:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916ba8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44422/0 (socket says 172.21.15.82:44422) 2023-07-14T19:35:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 -- 172.21.15.82:0/2303255600 learned_addr learned my addr 172.21.15.82:0/2303255600 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c5935700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916ba7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 -- 172.21.15.82:0/2303255600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 msgr2=0x559916ba7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916ba7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 -- 172.21.15.82:0/2303255600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 msgr2=0x559916ba8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916ba8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6136700 1 -- 172.21.15.82:0/2303255600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559916919d80 con 0x5599169f66b0 2023-07-14T19:35:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.059+0000 7ff5c6937700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916ba8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5c6136700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba76d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005d90 tx=0x7ff5b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f358f43cde3604 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:35:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/2303255600 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0024040 con 0x5599169f66b0 2023-07-14T19:35:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/2303255600 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7ff5b0017070 con 0x5599169f66b0 2023-07-14T19:35:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5b809f060 con 0x5599169f66b0 2023-07-14T19:35:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/2303255600 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b0004a30 con 0x5599169f66b0 2023-07-14T19:35:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 10 client.25811 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559916918250 con 0x5599169f66b0 2023-07-14T19:35:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/2303255600 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b809f060 con 0x5599169f66b0 2023-07-14T19:35:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/2303255600 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff5b0039020 con 0x5599169f66b0 2023-07-14T19:35:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 10 client.25811 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 20 client.25811 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916c08a10 0x559916c0ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 4 client.25811 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:35:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559916ae4e50 con 0x559916c08a10 2023-07-14T19:35:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.060+0000 7ff5c5935700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916c08a10 0x559916c0ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.061+0000 7ff5c5935700 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916c08a10 0x559916c0ae20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcec87ab65cf6d2d server_cookie=f299c982d29613da in_seq=0 out_seq=0 2023-07-14T19:35:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:38.061+0000 7ff5a9ffb700 10 client.25811 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:39.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:39 smithi133 ceph-mon[128894]: pgmap v219: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2023-07-14T19:35:39.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:39 smithi082 ceph-mon[127461]: pgmap v219: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2023-07-14T19:35:39.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:39 smithi161 ceph-mon[129184]: pgmap v219: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 351 op/s 2023-07-14T19:35:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5a9ffb700 1 -- 172.21.15.82:0/2303255600 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559916ae4e50 con 0x559916c08a10 2023-07-14T19:35:40.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5a9ffb700 10 client.25811 handle_command_reply: tid=0 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 client.25811 shutdown 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 msgr2=0x559916ba76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba76d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005d90 tx=0x7ff5b00047c0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 shutdown_connections 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559916ba0f00 0x559916ba8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599169f66b0 0x559916ba76d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x559916c08a10 0x559916c0ae20 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 --2- 172.21.15.82:0/2303255600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559916ba0a90 0x559916ba7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5a9ffb700 0 client.25811 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 >> 172.21.15.82:0/2303255600 conn(0x559916b903d0 msgr2=0x559916ba2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 shutdown_connections 2023-07-14T19:35:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.414+0000 7ff5d4532b80 1 -- 172.21.15.82:0/2303255600 wait complete. 2023-07-14T19:35:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5d4532b80 20 client.25811 trim_cache size 0 max 16384 2023-07-14T19:35:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:35:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:35:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:35:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:35:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5ac0a12c0 msgr2=0x7ff5ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5ac0a12c0 0x7ff5ac0a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005950 tx=0x7ff5b000b410 comp rx=0 tx=0).stop 2023-07-14T19:35:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 msgr2=0x7ff5c01a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c01a4060 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8007870 tx=0x7ff5b80078a0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 shutdown_connections 2023-07-14T19:35:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7ff5ac0a12c0 0x7ff5ac0a3770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5c0104fe0 0x7ff5c01a4060 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c00ffa10 0x7ff5c010c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 --2- 172.21.15.82:0/2582607110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c01003b0 0x7ff5c010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 >> 172.21.15.82:0/2582607110 conn(0x7ff5c0074cc0 msgr2=0x7ff5c01028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 shutdown_connections 2023-07-14T19:35:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:40.415+0000 7ff5c839a700 1 -- 172.21.15.82:0/2582607110 wait complete. 2023-07-14T19:35:41.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:41 smithi133 ceph-mon[128894]: pgmap v220: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 616 op/s 2023-07-14T19:35:41.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:41 smithi082 ceph-mon[127461]: pgmap v220: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 616 op/s 2023-07-14T19:35:41.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:41 smithi161 ceph-mon[129184]: pgmap v220: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 616 op/s 2023-07-14T19:35:43.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:43 smithi133 ceph-mon[128894]: pgmap v221: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 1.19k op/s 2023-07-14T19:35:43.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:43 smithi082 ceph-mon[127461]: pgmap v221: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 1.19k op/s 2023-07-14T19:35:43.500 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:43 smithi161 ceph-mon[129184]: pgmap v221: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 1.19k op/s 2023-07-14T19:35:45.442 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:45 smithi133 ceph-mon[128894]: pgmap v222: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.22k op/s 2023-07-14T19:35:45.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:45 smithi082 ceph-mon[127461]: pgmap v222: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.22k op/s 2023-07-14T19:35:45.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:45 smithi161 ceph-mon[129184]: pgmap v222: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.22k op/s 2023-07-14T19:35:47.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:47 smithi133 ceph-mon[128894]: pgmap v223: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 1.54k op/s 2023-07-14T19:35:47.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:47 smithi082 ceph-mon[127461]: pgmap v223: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 1.54k op/s 2023-07-14T19:35:47.500 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:47 smithi161 ceph-mon[129184]: pgmap v223: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 1.54k op/s 2023-07-14T19:35:49.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:49 smithi133 ceph-mon[128894]: pgmap v224: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 1.52k op/s 2023-07-14T19:35:49.473 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:49 smithi082 ceph-mon[127461]: pgmap v224: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 1.52k op/s 2023-07-14T19:35:49.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:49 smithi161 ceph-mon[129184]: pgmap v224: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 1.52k op/s 2023-07-14T19:35:51.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:51 smithi133 ceph-mon[128894]: pgmap v225: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 1.49k op/s 2023-07-14T19:35:51.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:51 smithi082 ceph-mon[127461]: pgmap v225: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 1.49k op/s 2023-07-14T19:35:51.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:51 smithi161 ceph-mon[129184]: pgmap v225: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 1.49k op/s 2023-07-14T19:35:53.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:53 smithi133 ceph-mon[128894]: pgmap v226: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 1.21k op/s 2023-07-14T19:35:53.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:53 smithi082 ceph-mon[127461]: pgmap v226: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 1.21k op/s 2023-07-14T19:35:53.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:53 smithi161 ceph-mon[129184]: pgmap v226: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 6.9 MiB/s wr, 1.21k op/s 2023-07-14T19:35:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 -- 172.21.15.82:0/2066603066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4178104ee0 msgr2=0x7f417810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 --2- 172.21.15.82:0/2066603066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4178104ee0 0x7f417810cab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f417000b0d0 tx=0x7f417000b3e0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 -- 172.21.15.82:0/2066603066 shutdown_connections 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 --2- 172.21.15.82:0/2066603066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4178104ee0 0x7f417810cab0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 --2- 172.21.15.82:0/2066603066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41780fe390 0x7f41780fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 --2- 172.21.15.82:0/2066603066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41780fed30 0x7f4178104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 -- 172.21.15.82:0/2066603066 >> 172.21.15.82:0/2066603066 conn(0x7f41780760c0 msgr2=0x7f41780764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 -- 172.21.15.82:0/2066603066 shutdown_connections 2023-07-14T19:35:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.527+0000 7f417df51700 1 -- 172.21.15.82:0/2066603066 wait complete. 2023-07-14T19:35:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 Processor -- start 2023-07-14T19:35:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 -- start start 2023-07-14T19:35:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41780fe390 0x7f4178195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 0x7f41781963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4178104ee0 0x7f417819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417819b060 con 0x7f41780fed30 2023-07-14T19:35:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417819b1d0 con 0x7f41780fe390 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f417df51700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417819b390 con 0x7f4178104ee0 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 0x7f41781963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 0x7f41781963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44434/0 (socket says 172.21.15.82:44434) 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f41777fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41780fe390 0x7f4178195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 -- 172.21.15.82:0/2673480279 learned_addr learned my addr 172.21.15.82:0/2673480279 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4177fff700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4178104ee0 0x7f417819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 -- 172.21.15.82:0/2673480279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4178104ee0 msgr2=0x7f417819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4178104ee0 0x7f417819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 -- 172.21.15.82:0/2673480279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41780fe390 msgr2=0x7f4178195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41780fe390 0x7f4178195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.528+0000 7f4176ffd700 1 -- 172.21.15.82:0/2673480279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170009d00 con 0x7f41780fed30 2023-07-14T19:35:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.529+0000 7f4176ffd700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 0x7f41781963b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f416800e940 tx=0x7f416800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.529+0000 7f4174ff9700 1 -- 172.21.15.82:0/2673480279 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168009730 con 0x7f41780fed30 2023-07-14T19:35:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.529+0000 7f4174ff9700 1 -- 172.21.15.82:0/2673480279 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4168010430 con 0x7f41780fed30 2023-07-14T19:35:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.529+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417819b670 con 0x7f41780fed30 2023-07-14T19:35:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.529+0000 7f4174ff9700 1 -- 172.21.15.82:0/2673480279 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416800cea0 con 0x7f41780fed30 2023-07-14T19:35:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.529+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41781a2eb0 con 0x7f41780fed30 2023-07-14T19:35:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.531+0000 7f4174ff9700 1 -- 172.21.15.82:0/2673480279 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4168010810 con 0x7f41780fed30 2023-07-14T19:35:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.531+0000 7f4174ff9700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f41600a12c0 0x7f41600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.531+0000 7f4174ff9700 1 -- 172.21.15.82:0/2673480279 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f4168014070 con 0x7f41780fed30 2023-07-14T19:35:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.531+0000 7f41777fe700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f41600a12c0 0x7f41600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f41777fe700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f41600a12c0 0x7f41600a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f416c009c60 tx=0x7f416c009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 Processor -- start 2023-07-14T19:35:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 -- start start 2023-07-14T19:35:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 0x55c1d311e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ea90 0x55c1d31189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ef00 0x55c1d3118f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d301dc50 con 0x55c1d2f746b0 2023-07-14T19:35:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d30709e0 con 0x55c1d311ef00 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d3069e70 con 0x55c1d311ea90 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f41777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 0x55c1d311e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f41777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 0x55c1d311e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44440/0 (socket says 172.21.15.82:44440) 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f41777fe700 1 -- 172.21.15.82:0/2048324553 learned_addr learned my addr 172.21.15.82:0/2048324553 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f4176ffd700 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ea90 0x55c1d31189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.532+0000 7f4177fff700 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ef00 0x55c1d3118f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f41777fe700 1 -- 172.21.15.82:0/2048324553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ea90 msgr2=0x55c1d31189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f41777fe700 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ea90 0x55c1d31189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f41777fe700 1 -- 172.21.15.82:0/2048324553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ef00 msgr2=0x55c1d3118f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f41777fe700 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ef00 0x55c1d3118f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f41777fe700 1 -- 172.21.15.82:0/2048324553 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1d2e97d80 con 0x55c1d2f746b0 2023-07-14T19:35:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f41777fe700 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 0x55c1d311e550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f416c00e4e0 tx=0x7f416c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ff75e55d2d3338 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f415dffb700 1 -- 172.21.15.82:0/2048324553 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00ee00 con 0x55c1d2f746b0 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f415dffb700 1 -- 172.21.15.82:0/2048324553 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f416c021070 con 0x55c1d2f746b0 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.533+0000 7f415dffb700 1 -- 172.21.15.82:0/2048324553 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c01d7b0 con 0x55c1d2f746b0 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2048324553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 msgr2=0x55c1d311e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 0x55c1d311e550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f416c00e4e0 tx=0x7f416c014000 comp rx=0 tx=0).stop 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2048324553 shutdown_connections 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ef00 0x55c1d3118f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d2f746b0 0x55c1d311e550 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2048324553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ea90 0x55c1d31189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2048324553 >> 172.21.15.82:0/2048324553 conn(0x55c1d310e3d0 msgr2=0x55c1d3113530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2048324553 shutdown_connections 2023-07-14T19:35:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2048324553 wait complete. 2023-07-14T19:35:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 -- start start 2023-07-14T19:35:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.534+0000 7f418a0e9b80 1 Processor -- start 2023-07-14T19:35:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d3127de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d3128320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d3009a60 con 0x55c1d311ef00 2023-07-14T19:35:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d3009760 con 0x55c1d311ea90 2023-07-14T19:35:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d2eed8c0 con 0x55c1d2f746b0 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 10 client.0 _ll_register_callbacks cb 0x55c1d3007700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f41777fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d3127de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31220a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44450/0 (socket says 172.21.15.82:44450) 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 -- 172.21.15.82:0/1064040754 learned_addr learned my addr 172.21.15.82:0/1064040754 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4176ffd700 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d3128320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 -- 172.21.15.82:0/1064040754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 msgr2=0x55c1d3127de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d3127de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 -- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 msgr2=0x55c1d3128320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d3128320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 -- 172.21.15.82:0/1064040754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c017070 con 0x55c1d311ef00 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4177fff700 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31220a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4170008f80 tx=0x7f4170008fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23518f5cf1f06dc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4156ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:35:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4156ffd700 1 -- 172.21.15.82:0/1064040754 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000e040 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4156ffd700 1 -- 172.21.15.82:0/1064040754 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f41700074e0 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1d2fc3490 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f4156ffd700 1 -- 172.21.15.82:0/1064040754 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41700076e0 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 10 client.16116 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.535+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c1d2e80c00 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f4156ffd700 1 -- 172.21.15.82:0/1064040754 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1d2fc3490 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f4156ffd700 1 -- 172.21.15.82:0/1064040754 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f417002d050 con 0x55c1d311ef00 2023-07-14T19:35:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f418a0e9b80 10 client.16116 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f418a0e9b80 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f418a0e9b80 20 client.16116 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d31322c0 0x55c1d31346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f418a0e9b80 4 client.16116 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1d30e7d40 con 0x55c1d31322c0 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f41777fe700 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d31322c0 0x55c1d31346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f4156ffd700 10 client.16116 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.536+0000 7f41777fe700 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d31322c0 0x55c1d31346d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c93869dc089293b server_cookie=afa6814efa4f8f33 in_seq=0 out_seq=0 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.537+0000 7f4156ffd700 1 -- 172.21.15.82:0/1064040754 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c1d30e7d40 con 0x55c1d31322c0 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.537+0000 7f4156ffd700 10 client.16116 handle_command_reply: tid=0 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.537+0000 7f418a0e9b80 1 client.16116 shutdown 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.537+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 msgr2=0x55c1d31220a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.537+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31220a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4170008f80 tx=0x7f4170008fb0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 shutdown_connections 2023-07-14T19:35:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31220a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d31322c0 0x55c1d31346d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d3127de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1064040754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d3128320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f4156ffd700 0 client.16116 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 >> 172.21.15.82:0/1064040754 conn(0x55c1d310e3d0 msgr2=0x55c1d3112c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 shutdown_connections 2023-07-14T19:35:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1064040754 wait complete. 2023-07-14T19:35:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.538+0000 7f418a0e9b80 20 client.16116 trim_cache size 0 max 16384 2023-07-14T19:35:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 Processor -- start 2023-07-14T19:35:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 -- start start 2023-07-14T19:35:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d311d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d31133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d3119a00 con 0x55c1d311ef00 2023-07-14T19:35:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d3114050 con 0x55c1d311ea90 2023-07-14T19:35:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d3120650 con 0x55c1d2f746b0 2023-07-14T19:35:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4176ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d31133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31189e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44458/0 (socket says 172.21.15.82:44458) 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f41777fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d311d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 -- 172.21.15.82:0/2526174308 learned_addr learned my addr 172.21.15.82:0/2526174308 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 -- 172.21.15.82:0/2526174308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 msgr2=0x55c1d311d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d311d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 -- 172.21.15.82:0/2526174308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 msgr2=0x55c1d31133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d31133b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 -- 172.21.15.82:0/2526174308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1d2ebe1f0 con 0x55c1d311ef00 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.553+0000 7f4177fff700 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31189e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f417000b010 tx=0x7f417000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ba5b95ecbdf00e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f415e7fc700 1 -- 172.21.15.82:0/2526174308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417000e040 con 0x55c1d311ef00 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f415e7fc700 1 -- 172.21.15.82:0/2526174308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4170003a00 con 0x55c1d311ef00 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f415e7fc700 1 -- 172.21.15.82:0/2526174308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700123f0 con 0x55c1d311ef00 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2526174308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 msgr2=0x55c1d31189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31189e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f417000b010 tx=0x7f417000b730 comp rx=0 tx=0).stop 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2526174308 shutdown_connections 2023-07-14T19:35:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ef00 0x55c1d31189e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d2f746b0 0x55c1d311d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/2526174308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d311ea90 0x55c1d31133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2526174308 >> 172.21.15.82:0/2526174308 conn(0x55c1d310e3d0 msgr2=0x55c1d311d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2526174308 shutdown_connections 2023-07-14T19:35:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.554+0000 7f418a0e9b80 1 -- 172.21.15.82:0/2526174308 wait complete. 2023-07-14T19:35:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 -- start start 2023-07-14T19:35:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 Processor -- start 2023-07-14T19:35:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 0x55c1d312e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ea90 0x55c1d312ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ef00 0x55c1d312b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d312b620 con 0x55c1d311ea90 2023-07-14T19:35:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d312b760 con 0x55c1d2f746b0 2023-07-14T19:35:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1d312b920 con 0x55c1d311ef00 2023-07-14T19:35:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f418a0e9b80 10 client.0 _ll_register_callbacks cb 0x7f4170009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:35:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f41777fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 0x55c1d312e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f4176ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ea90 0x55c1d312ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f41777fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 0x55c1d312e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40808/0 (socket says 172.21.15.82:40808) 2023-07-14T19:35:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f41777fe700 1 -- 172.21.15.82:0/1396592672 learned_addr learned my addr 172.21.15.82:0/1396592672 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.555+0000 7f4177fff700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ef00 0x55c1d312b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f41777fe700 1 -- 172.21.15.82:0/1396592672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ef00 msgr2=0x55c1d312b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f41777fe700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ef00 0x55c1d312b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f41777fe700 1 -- 172.21.15.82:0/1396592672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ea90 msgr2=0x55c1d312ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f41777fe700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ea90 0x55c1d312ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f41777fe700 1 -- 172.21.15.82:0/1396592672 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1d2e80c00 con 0x55c1d2f746b0 2023-07-14T19:35:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4176ffd700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ea90 0x55c1d312ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4177fff700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ef00 0x55c1d312b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f41777fe700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 0x55c1d312e680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f416c00e4b0 tx=0x7f416c0091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c7c03cd91f3d33 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4156ffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4156ffd700 1 -- 172.21.15.82:0/1396592672 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00ecb0 con 0x55c1d2f746b0 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4156ffd700 1 -- 172.21.15.82:0/1396592672 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f416c0330b0 con 0x55c1d2f746b0 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f416c021070 con 0x55c1d2f746b0 2023-07-14T19:35:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4156ffd700 1 -- 172.21.15.82:0/1396592672 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f416c033270 con 0x55c1d2f746b0 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f418a0e9b80 10 client.25829 fetch_fsmap learned FSMap version 42 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c1d2e96250 con 0x55c1d2f746b0 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.556+0000 7f4156ffd700 1 -- 172.21.15.82:0/1396592672 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f416c021070 con 0x55c1d2f746b0 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f4156ffd700 1 -- 172.21.15.82:0/1396592672 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 42) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f416c03a060 con 0x55c1d2f746b0 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f418a0e9b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 42 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f418a0e9b80 10 client.25829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f418a0e9b80 20 client.25829 populate_metadata read hostname 'smithi082' 2023-07-14T19:35:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d318b920 0x55c1d318dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:35:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f418a0e9b80 4 client.25829 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:35:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c1d3062e50 con 0x55c1d318b920 2023-07-14T19:35:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f4176ffd700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d318b920 0x55c1d318dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:35:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f4176ffd700 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d318b920 0x55c1d318dd30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4d914255c3d20a server_cookie=e377af637f5c58b9 in_seq=0 out_seq=0 2023-07-14T19:35:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:53.557+0000 7f4156ffd700 10 client.25829 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f4156ffd700 1 -- 172.21.15.82:0/1396592672 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c1d3062e50 con 0x55c1d318b920 2023-07-14T19:35:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f4156ffd700 10 client.25829 handle_command_reply: tid=0 2023-07-14T19:35:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 client.25829 shutdown 2023-07-14T19:35:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 msgr2=0x55c1d312e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 0x55c1d312e680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f416c00e4b0 tx=0x7f416c0091e0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 shutdown_connections 2023-07-14T19:35:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c1d318b920 0x55c1d318dd30 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c1d311ef00 0x55c1d312b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1d2f746b0 0x55c1d312e680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 --2- 172.21.15.82:0/1396592672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1d311ea90 0x55c1d312ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 >> 172.21.15.82:0/1396592672 conn(0x55c1d310e3d0 msgr2=0x55c1d311d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f4156ffd700 0 client.25829 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:35:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 shutdown_connections 2023-07-14T19:35:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.455+0000 7f418a0e9b80 1 -- 172.21.15.82:0/1396592672 wait complete. 2023-07-14T19:35:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.456+0000 7f418a0e9b80 20 client.25829 trim_cache size 0 max 16384 2023-07-14T19:35:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:35:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:35:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:35:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:35:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f41600a12c0 msgr2=0x7f41600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f41600a12c0 0x7f41600a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f416c009c60 tx=0x7f416c009260 comp rx=0 tx=0).stop 2023-07-14T19:35:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 msgr2=0x7f41781963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:35:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 0x7f41781963b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f416800e940 tx=0x7f416800ec50 comp rx=0 tx=0).stop 2023-07-14T19:35:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 shutdown_connections 2023-07-14T19:35:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f41600a12c0 0x7f41600a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4178104ee0 0x7f417819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41780fe390 0x7f4178195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 --2- 172.21.15.82:0/2673480279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41780fed30 0x7f41781963b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:35:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.457+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 >> 172.21.15.82:0/2673480279 conn(0x7f41780760c0 msgr2=0x7f41780fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:35:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.458+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 shutdown_connections 2023-07-14T19:35:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:35:54.458+0000 7f417df51700 1 -- 172.21.15.82:0/2673480279 wait complete. 2023-07-14T19:35:55.067 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:35:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: ignoring --setuser ceph since I am not root 2023-07-14T19:35:55.067 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:35:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: ignoring --setgroup ceph since I am not root 2023-07-14T19:35:55.067 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:35:54 smithi161 ceph-mds[159768]: -- 172.21.15.161:0/2729896067 <== mon.2 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac236231e0 con 0x55ac235f8800 2023-07-14T19:35:55.067 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:35:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: starting mds.c at 2023-07-14T19:35:55.068 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:55 smithi161 ceph-mon[129184]: pgmap v227: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 583 op/s 2023-07-14T19:35:55.068 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:55 smithi161 ceph-mon[129184]: daemon mds.c restarted 2023-07-14T19:35:55.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:55 smithi133 ceph-mon[128894]: pgmap v227: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 583 op/s 2023-07-14T19:35:55.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:55 smithi133 ceph-mon[128894]: daemon mds.c restarted 2023-07-14T19:35:55.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:55 smithi082 ceph-mon[127461]: pgmap v227: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 583 op/s 2023-07-14T19:35:55.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:55 smithi082 ceph-mon[127461]: daemon mds.c restarted 2023-07-14T19:35:56.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:56 smithi133 ceph-mon[128894]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T19:35:56.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:56 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6836/3620688590,v1:172.21.15.161:6837/3620688590] up:boot 2023-07-14T19:35:56.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:56 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:35:56.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:56 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:35:56.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:56 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:56.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:56 smithi082 ceph-mon[127461]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T19:35:56.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:56 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6836/3620688590,v1:172.21.15.161:6837/3620688590] up:boot 2023-07-14T19:35:56.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:56 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:35:56.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:56 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:35:56.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:56 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:56.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:56 smithi161 ceph-mon[129184]: osdmap e89: 12 total, 12 up, 12 in 2023-07-14T19:35:56.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:56 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6836/3620688590,v1:172.21.15.161:6837/3620688590] up:boot 2023-07-14T19:35:56.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:56 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-14T19:35:56.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:56 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:35:56.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:56 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:35:57.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:57 smithi133 ceph-mon[128894]: pgmap v229: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 419 op/s 2023-07-14T19:35:57.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:57 smithi082 ceph-mon[127461]: pgmap v229: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 419 op/s 2023-07-14T19:35:57.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:57 smithi161 ceph-mon[129184]: pgmap v229: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 419 op/s 2023-07-14T19:35:59.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:35:59 smithi133 ceph-mon[128894]: pgmap v230: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 499 op/s 2023-07-14T19:35:59.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:35:59 smithi082 ceph-mon[127461]: pgmap v230: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 499 op/s 2023-07-14T19:35:59.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:35:59 smithi161 ceph-mon[129184]: pgmap v230: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 10 MiB/s wr, 499 op/s 2023-07-14T19:36:01.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:01 smithi133 ceph-mon[128894]: pgmap v231: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.6 MiB/s wr, 538 op/s 2023-07-14T19:36:01.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:01 smithi082 ceph-mon[127461]: pgmap v231: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.6 MiB/s wr, 538 op/s 2023-07-14T19:36:01.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:01 smithi161 ceph-mon[129184]: pgmap v231: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.6 MiB/s wr, 538 op/s 2023-07-14T19:36:03.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:03 smithi133 ceph-mon[128894]: pgmap v232: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 556 op/s 2023-07-14T19:36:03.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:03 smithi082 ceph-mon[127461]: pgmap v232: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 556 op/s 2023-07-14T19:36:03.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:03 smithi161 ceph-mon[129184]: pgmap v232: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 556 op/s 2023-07-14T19:36:05.443 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:05 smithi133 ceph-mon[128894]: pgmap v233: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 556 op/s 2023-07-14T19:36:05.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:05 smithi082 ceph-mon[127461]: pgmap v233: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 556 op/s 2023-07-14T19:36:05.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:05 smithi161 ceph-mon[129184]: pgmap v233: 97 pgs: 97 active+clean; 1.7 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.9 MiB/s wr, 556 op/s 2023-07-14T19:36:07.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:07 smithi133 ceph-mon[128894]: pgmap v234: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2023-07-14T19:36:07.474 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:07 smithi082 ceph-mon[127461]: pgmap v234: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2023-07-14T19:36:07.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:07 smithi161 ceph-mon[129184]: pgmap v234: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 375 op/s 2023-07-14T19:36:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 -- 172.21.15.82:0/2168979786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e40fed30 msgr2=0x7f53e4104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 --2- 172.21.15.82:0/2168979786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e40fed30 0x7f53e4104800 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f53e00099c0 tx=0x7f53e0009cd0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 -- 172.21.15.82:0/2168979786 shutdown_connections 2023-07-14T19:36:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 --2- 172.21.15.82:0/2168979786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e4104ee0 0x7f53e410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 --2- 172.21.15.82:0/2168979786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e40fe390 0x7f53e40fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 --2- 172.21.15.82:0/2168979786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e40fed30 0x7f53e4104800 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 -- 172.21.15.82:0/2168979786 >> 172.21.15.82:0/2168979786 conn(0x7f53e40760c0 msgr2=0x7f53e40764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 -- 172.21.15.82:0/2168979786 shutdown_connections 2023-07-14T19:36:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.587+0000 7f53ec709700 1 -- 172.21.15.82:0/2168979786 wait complete. 2023-07-14T19:36:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 Processor -- start 2023-07-14T19:36:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 -- start start 2023-07-14T19:36:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e40fe390 0x7f53e4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e40fed30 0x7f53e4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 0x7f53e419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419d120 con 0x7f53e4104ee0 2023-07-14T19:36:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419d290 con 0x7f53e40fed30 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.588+0000 7f53ec709700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e419d450 con 0x7f53e40fe390 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 0x7f53e419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e40fed30 0x7f53e4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 0x7f53e419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44478/0 (socket says 172.21.15.82:44478) 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e40fed30 0x7f53e4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40824/0 (socket says 172.21.15.82:40824) 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 -- 172.21.15.82:0/3127959636 learned_addr learned my addr 172.21.15.82:0/3127959636 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e40fe390 0x7f53e4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 -- 172.21.15.82:0/3127959636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e40fe390 msgr2=0x7f53e4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e40fe390 0x7f53e4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 -- 172.21.15.82:0/3127959636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e40fed30 msgr2=0x7f53e4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e40fed30 0x7f53e4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 -- 172.21.15.82:0/3127959636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00095f0 con 0x7f53e4104ee0 2023-07-14T19:36:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53eaca6700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 0x7f53e419cae0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00ef30 tx=0x7f53dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53db7fe700 1 -- 172.21.15.82:0/3127959636 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc018070 con 0x7f53e4104ee0 2023-07-14T19:36:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e419d6d0 con 0x7f53e4104ee0 2023-07-14T19:36:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53db7fe700 1 -- 172.21.15.82:0/3127959636 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53dc00f040 con 0x7f53e4104ee0 2023-07-14T19:36:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53db7fe700 1 -- 172.21.15.82:0/3127959636 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc00cd00 con 0x7f53e4104ee0 2023-07-14T19:36:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.589+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e41a4f70 con 0x7f53e4104ee0 2023-07-14T19:36:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.592+0000 7f53db7fe700 1 -- 172.21.15.82:0/3127959636 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f53dc003a00 con 0x7f53e4104ee0 2023-07-14T19:36:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.592+0000 7f53db7fe700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f53d00a12c0 0x7f53d00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.592+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f53d00a12c0 0x7f53d00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.592+0000 7f53db7fe700 1 -- 172.21.15.82:0/3127959636 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f53dc0cc740 con 0x7f53e4104ee0 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f53d00a12c0 0x7f53d00a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009c60 tx=0x7f53d4009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 Processor -- start 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 -- start start 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 0x55817d369550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d3639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369f00 0x55817d363f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d268c50 con 0x55817d1bf6b0 2023-07-14T19:36:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d2bb9e0 con 0x55817d369a90 2023-07-14T19:36:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.593+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d2b4e70 con 0x55817d369f00 2023-07-14T19:36:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53ea4a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 0x55817d369550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d3639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53ea4a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 0x55817d369550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44482/0 (socket says 172.21.15.82:44482) 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d3639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40832/0 (socket says 172.21.15.82:40832) 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53ea4a5700 1 -- 172.21.15.82:0/1938722827 learned_addr learned my addr 172.21.15.82:0/1938722827 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369f00 0x55817d363f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 -- 172.21.15.82:0/1938722827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 msgr2=0x55817d3639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d3639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 -- 172.21.15.82:0/1938722827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 msgr2=0x55817d369550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 0x55817d369550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 -- 172.21.15.82:0/1938722827 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817d0e2d80 con 0x55817d369f00 2023-07-14T19:36:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 0x55817d369550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53eaca6700 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369f00 0x55817d363f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0c9f20 tx=0x7f53dc099470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfbbd0dd082e8a64 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53d8ff9700 1 -- 172.21.15.82:0/1938722827 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc084040 con 0x55817d369f00 2023-07-14T19:36:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53d8ff9700 1 -- 172.21.15.82:0/1938722827 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53dc004ad0 con 0x55817d369f00 2023-07-14T19:36:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.594+0000 7f53d8ff9700 1 -- 172.21.15.82:0/1938722827 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc099d90 con 0x55817d369f00 2023-07-14T19:36:07.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1938722827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369f00 msgr2=0x55817d363f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369f00 0x55817d363f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0c9f20 tx=0x7f53dc099470 comp rx=0 tx=0).stop 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1938722827 shutdown_connections 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369f00 0x55817d363f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d1bf6b0 0x55817d369550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1938722827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d3639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1938722827 >> 172.21.15.82:0/1938722827 conn(0x55817d3593d0 msgr2=0x55817d35e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1938722827 shutdown_connections 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1938722827 wait complete. 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 -- start start 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.595+0000 7f53f88a1b80 1 Processor -- start 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d373360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d36d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d254a60 con 0x55817d369f00 2023-07-14T19:36:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d254760 con 0x55817d1bf6b0 2023-07-14T19:36:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d1388c0 con 0x55817d369a90 2023-07-14T19:36:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 10 client.0 _ll_register_callbacks cb 0x55817d252700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53eaca6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d36d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40836/0 (socket says 172.21.15.82:40836) 2023-07-14T19:36:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 -- 172.21.15.82:0/113738066 learned_addr learned my addr 172.21.15.82:0/113738066 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d373360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 -- 172.21.15.82:0/113738066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 msgr2=0x55817d373360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d373360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 -- 172.21.15.82:0/113738066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 msgr2=0x55817d36d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d36d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 -- 172.21.15.82:0/113738066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d4017070 con 0x55817d1bf6b0 2023-07-14T19:36:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53eaca6700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d36d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f53d4005c50 tx=0x7f53d40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d433bcd1d101a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53cdffb700 1 -- 172.21.15.82:0/113738066 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4022070 con 0x55817d1bf6b0 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.596+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55817d20e490 con 0x55817d1bf6b0 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53cdffb700 1 -- 172.21.15.82:0/113738066 <== mon.2 v2:172.21.15.133:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f53d401d6e0 con 0x55817d1bf6b0 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53cdffb700 1 -- 172.21.15.82:0/113738066 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53d401d8a0 con 0x55817d1bf6b0 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 10 client.25847 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f53e41a4f70 con 0x55817d1bf6b0 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53cdffb700 1 -- 172.21.15.82:0/113738066 <== mon.2 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55817d20e490 con 0x55817d1bf6b0 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53cdffb700 1 -- 172.21.15.82:0/113738066 <== mon.2 v2:172.21.15.133:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f53d4027a30 con 0x55817d1bf6b0 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 10 client.25847 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 10 client.25847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 20 client.25847 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d37d3f0 0x55817d37f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 4 client.25847 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:36:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55817d332d40 con 0x55817d37d3f0 2023-07-14T19:36:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.597+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d37d3f0 0x55817d37f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.598+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d37d3f0 0x55817d37f800 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a2f2d6115efcc2 server_cookie=6a3d67d521e3b7ca in_seq=0 out_seq=0 2023-07-14T19:36:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.598+0000 7f53cdffb700 10 client.25847 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.598+0000 7f53cdffb700 1 -- 172.21.15.82:0/113738066 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55817d332d40 con 0x55817d37d3f0 2023-07-14T19:36:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.598+0000 7f53cdffb700 10 client.25847 handle_command_reply: tid=0 2023-07-14T19:36:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 client.25847 shutdown 2023-07-14T19:36:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 msgr2=0x55817d372e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f53d4005c50 tx=0x7f53d40047c0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 shutdown_connections 2023-07-14T19:36:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d36d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372e20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d37d3f0 0x55817d37f800 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/113738066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d373360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53cdffb700 0 client.25847 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 >> 172.21.15.82:0/113738066 conn(0x55817d3593d0 msgr2=0x55817d35dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 shutdown_connections 2023-07-14T19:36:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 1 -- 172.21.15.82:0/113738066 wait complete. 2023-07-14T19:36:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.599+0000 7f53f88a1b80 20 client.25847 trim_cache size 0 max 16384 2023-07-14T19:36:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 Processor -- start 2023-07-14T19:36:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 -- start start 2023-07-14T19:36:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d1bf6b0 0x55817d36d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d35e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d368920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d35ff30 con 0x55817d369f00 2023-07-14T19:36:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d364970 con 0x55817d369a90 2023-07-14T19:36:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d367e00 con 0x55817d1bf6b0 2023-07-14T19:36:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.619+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d35e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d35e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40846/0 (socket says 172.21.15.82:40846) 2023-07-14T19:36:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 -- 172.21.15.82:0/2540260057 learned_addr learned my addr 172.21.15.82:0/2540260057 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d1bf6b0 0x55817d36d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 -- 172.21.15.82:0/2540260057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d1bf6b0 msgr2=0x55817d36d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d1bf6b0 0x55817d36d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 -- 172.21.15.82:0/2540260057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 msgr2=0x55817d368920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:36:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d368920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 -- 172.21.15.82:0/2540260057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817d0cbc00 con 0x55817d369a90 2023-07-14T19:36:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d35e3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f53e0000c00 tx=0x7f53e0009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2150c3fc75a34050 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.620+0000 7f53d97fa700 1 -- 172.21.15.82:0/2540260057 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00282b0 con 0x55817d369a90 2023-07-14T19:36:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.621+0000 7f53d97fa700 1 -- 172.21.15.82:0/2540260057 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e001e040 con 0x55817d369a90 2023-07-14T19:36:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.621+0000 7f53d97fa700 1 -- 172.21.15.82:0/2540260057 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0003c30 con 0x55817d369a90 2023-07-14T19:36:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 -- 172.21.15.82:0/2540260057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 msgr2=0x55817d35e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d35e3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f53e0000c00 tx=0x7f53e0009510 comp rx=0 tx=0).stop 2023-07-14T19:36:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 -- 172.21.15.82:0/2540260057 shutdown_connections 2023-07-14T19:36:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d368920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d1bf6b0 0x55817d36d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/2540260057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d369a90 0x55817d35e3b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.622+0000 7f53f88a1b80 1 -- 172.21.15.82:0/2540260057 >> 172.21.15.82:0/2540260057 conn(0x55817d3593d0 msgr2=0x55817d368210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.623+0000 7f53f88a1b80 1 -- 172.21.15.82:0/2540260057 shutdown_connections 2023-07-14T19:36:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 -- 172.21.15.82:0/2540260057 wait complete. 2023-07-14T19:36:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 -- start start 2023-07-14T19:36:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 Processor -- start 2023-07-14T19:36:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d371130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d371670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d371bb0 con 0x55817d369f00 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d371d20 con 0x55817d1bf6b0 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.625+0000 7f53f88a1b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55817d371ee0 con 0x55817d369a90 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53f88a1b80 10 client.0 _ll_register_callbacks cb 0x7f53d4004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d371130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d371130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53766/0 (socket says 172.21.15.82:53766) 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 -- 172.21.15.82:0/1158398241 learned_addr learned my addr 172.21.15.82:0/1158398241 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53eaca6700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d371670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 -- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 msgr2=0x55817d372eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 -- 172.21.15.82:0/1158398241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 msgr2=0x55817d371670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d371670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 -- 172.21.15.82:0/1158398241 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e410d2f0 con 0x55817d369a90 2023-07-14T19:36:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:36:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53e9ca4700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d371130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53e00095c0 tx=0x7f53e0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d938033b73dbd100 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:36:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.626+0000 7f53cdffb700 1 -- 172.21.15.82:0/1158398241 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0028c20 con 0x55817d369a90 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.627+0000 7f53cdffb700 1 -- 172.21.15.82:0/1158398241 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8382+0+0 (secure 0 0 0) 0x7f53e0028d80 con 0x55817d369a90 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.627+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53dc004ad0 con 0x55817d369a90 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.627+0000 7f53cdffb700 1 -- 172.21.15.82:0/1158398241 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e00037d0 con 0x55817d369a90 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.627+0000 7f53f88a1b80 10 client.25696 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.627+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55817d1091f0 con 0x55817d369a90 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53cdffb700 1 -- 172.21.15.82:0/1158398241 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53dc004ad0 con 0x55817d369a90 2023-07-14T19:36:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53cdffb700 1 -- 172.21.15.82:0/1158398241 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f53e000bc20 con 0x55817d369a90 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53f88a1b80 10 client.25696 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53f88a1b80 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53f88a1b80 20 client.25696 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d3cbf60 0x55817d3cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53f88a1b80 4 client.25696 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.629+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55817d2ade50 con 0x55817d3cbf60 2023-07-14T19:36:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.630+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d3cbf60 0x55817d3cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.631+0000 7f53ea4a5700 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d3cbf60 0x55817d3cc360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a046332e410ea9 server_cookie=2b24d0cdcf7e3374 in_seq=0 out_seq=0 2023-07-14T19:36:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.631+0000 7f53cdffb700 10 client.25696 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53cdffb700 1 -- 172.21.15.82:0/1158398241 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55817d2ade50 con 0x55817d3cbf60 2023-07-14T19:36:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53cdffb700 10 client.25696 handle_command_reply: tid=0 2023-07-14T19:36:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 client.25696 shutdown 2023-07-14T19:36:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 msgr2=0x55817d371130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d371130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f53e00095c0 tx=0x7f53e0009310 comp rx=0 tx=0).stop 2023-07-14T19:36:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 shutdown_connections 2023-07-14T19:36:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55817d369f00 0x55817d371670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55817d1bf6b0 0x55817d372eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55817d3cbf60 0x55817d3cc360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 --2- 172.21.15.82:0/1158398241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55817d369a90 0x55817d371130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 >> 172.21.15.82:0/1158398241 conn(0x55817d3593d0 msgr2=0x55817d368910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.685+0000 7f53cdffb700 0 client.25696 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:07.699 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.686+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 shutdown_connections 2023-07-14T19:36:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.686+0000 7f53f88a1b80 1 -- 172.21.15.82:0/1158398241 wait complete. 2023-07-14T19:36:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.688+0000 7f53f88a1b80 20 client.25696 trim_cache size 0 max 16384 2023-07-14T19:36:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:36:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:36:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:36:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:36:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.690+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f53d00a12c0 msgr2=0x7f53d00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.690+0000 7f53ec709700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f53d00a12c0 0x7f53d00a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f53d4009c60 tx=0x7f53d4009260 comp rx=0 tx=0).stop 2023-07-14T19:36:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.690+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 msgr2=0x7f53e419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.690+0000 7f53ec709700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 0x7f53e419cae0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00ef30 tx=0x7f53dc00c3a0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 shutdown_connections 2023-07-14T19:36:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f53d00a12c0 0x7f53d00a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53e4104ee0 0x7f53e419cae0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53e40fe390 0x7f53e4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 --2- 172.21.15.82:0/3127959636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53e40fed30 0x7f53e4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 >> 172.21.15.82:0/3127959636 conn(0x7f53e40760c0 msgr2=0x7f53e40fc030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 shutdown_connections 2023-07-14T19:36:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:07.692+0000 7f53ec709700 1 -- 172.21.15.82:0/3127959636 wait complete. 2023-07-14T19:36:09.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:09 smithi133 ceph-mon[128894]: pgmap v235: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2023-07-14T19:36:09.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:09 smithi082 ceph-mon[127461]: pgmap v235: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2023-07-14T19:36:09.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:09 smithi161 ceph-mon[129184]: pgmap v235: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 337 op/s 2023-07-14T19:36:11.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:11 smithi133 ceph-mon[128894]: pgmap v236: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2023-07-14T19:36:11.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:11 smithi082 ceph-mon[127461]: pgmap v236: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2023-07-14T19:36:11.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:11 smithi161 ceph-mon[129184]: pgmap v236: 97 pgs: 97 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 298 op/s 2023-07-14T19:36:12.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:12 smithi133 ceph-mon[128894]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T19:36:12.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:12 smithi082 ceph-mon[127461]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T19:36:12.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:12 smithi161 ceph-mon[129184]: osdmap e90: 12 total, 12 up, 12 in 2023-07-14T19:36:13.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:13 smithi133 ceph-mon[128894]: pgmap v238: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-07-14T19:36:13.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:13 smithi082 ceph-mon[127461]: pgmap v238: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-07-14T19:36:13.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:13 smithi161 ceph-mon[129184]: pgmap v238: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 290 op/s 2023-07-14T19:36:15.450 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:15 smithi133 ceph-mon[128894]: pgmap v239: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2023-07-14T19:36:15.451 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:15 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:36:15.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:15 smithi082 ceph-mon[127461]: pgmap v239: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2023-07-14T19:36:15.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:15 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:36:15.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:15 smithi161 ceph-mon[129184]: pgmap v239: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.7 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2023-07-14T19:36:15.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:15 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:36:16.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:16 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:36:16.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:16 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:36:16.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:16 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:36:16.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:16 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:36:16.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:16 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:36:16.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:16 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:36:16.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:16 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:36:16.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:16 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:36:16.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:16 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:36:17.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:17 smithi133 ceph-mon[128894]: pgmap v240: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 371 op/s 2023-07-14T19:36:17.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:17 smithi082 ceph-mon[127461]: pgmap v240: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 371 op/s 2023-07-14T19:36:17.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:17 smithi161 ceph-mon[129184]: pgmap v240: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 371 op/s 2023-07-14T19:36:19.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:19 smithi133 ceph-mon[128894]: pgmap v241: 97 pgs: 31 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 50 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 401 op/s 2023-07-14T19:36:19.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:19 smithi082 ceph-mon[127461]: pgmap v241: 97 pgs: 31 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 50 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 401 op/s 2023-07-14T19:36:19.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:19 smithi161 ceph-mon[129184]: pgmap v241: 97 pgs: 31 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 50 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 401 op/s 2023-07-14T19:36:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.848+0000 7f0883c23700 1 -- 172.21.15.82:0/1211846741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c104b80 msgr2=0x7f087c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.848+0000 7f0883c23700 1 --2- 172.21.15.82:0/1211846741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c104b80 0x7f087c104f50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878009ca0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 -- 172.21.15.82:0/1211846741 shutdown_connections 2023-07-14T19:36:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 --2- 172.21.15.82:0/1211846741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087c0ff2e0 0x7f087c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 --2- 172.21.15.82:0/1211846741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c104b80 0x7f087c104f50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 --2- 172.21.15.82:0/1211846741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c105490 0x7f087c0febb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 -- 172.21.15.82:0/1211846741 >> 172.21.15.82:0/1211846741 conn(0x7f087c076ae0 msgr2=0x7f087c076ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 -- 172.21.15.82:0/1211846741 shutdown_connections 2023-07-14T19:36:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 -- 172.21.15.82:0/1211846741 wait complete. 2023-07-14T19:36:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 Processor -- start 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.849+0000 7f0883c23700 1 -- start start 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f0883c23700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c0ff2e0 0x7f087c194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f0883c23700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087c104b80 0x7f087c1951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f0883c23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 0x7f087c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f0883c23700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c198db0 con 0x7f087c105490 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f0883c23700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c198f20 con 0x7f087c104b80 2023-07-14T19:36:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f0883c23700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c1990e0 con 0x7f087c0ff2e0 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08821c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 0x7f087c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c0ff2e0 0x7f087c194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08821c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 0x7f087c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44518/0 (socket says 172.21.15.82:44518) 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08821c0700 1 -- 172.21.15.82:0/1397311883 learned_addr learned my addr 172.21.15.82:0/1397311883 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08811be700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087c104b80 0x7f087c1951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 -- 172.21.15.82:0/1397311883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087c104b80 msgr2=0x7f087c1951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087c104b80 0x7f087c1951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 -- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 msgr2=0x7f087c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 0x7f087c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 -- 172.21.15.82:0/1397311883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08780095f0 con 0x7f087c0ff2e0 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08821c0700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 0x7f087c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.850+0000 7f08819bf700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c0ff2e0 0x7f087c194cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.851+0000 7f0872ffd700 1 -- 172.21.15.82:0/1397311883 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087801d070 con 0x7f087c0ff2e0 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.851+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c199360 con 0x7f087c0ff2e0 2023-07-14T19:36:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.851+0000 7f0872ffd700 1 -- 172.21.15.82:0/1397311883 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0878021410 con 0x7f087c0ff2e0 2023-07-14T19:36:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.851+0000 7f0872ffd700 1 -- 172.21.15.82:0/1397311883 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878017650 con 0x7f087c0ff2e0 2023-07-14T19:36:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.851+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c199630 con 0x7f087c0ff2e0 2023-07-14T19:36:20.870 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.853+0000 7f0872ffd700 1 -- 172.21.15.82:0/1397311883 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0878021810 con 0x7f087c0ff2e0 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.854+0000 7f0872ffd700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f08680a12c0 0x7f08680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.854+0000 7f08811be700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f08680a12c0 0x7f08680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.854+0000 7f0872ffd700 1 -- 172.21.15.82:0/1397311883 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f08780cdff0 con 0x7f087c0ff2e0 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.854+0000 7f08811be700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f08680a12c0 0x7f08680a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f087c196490 tx=0x7f086c009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.854+0000 7f088fdbbb80 1 Processor -- start 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 -- start start 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd257546b0 0x55bd258f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.871 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 0x55bd258fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd25906e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd257fdc50 con 0x55bd258f9cc0 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd258509e0 con 0x55bd25904a40 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd25849e70 con 0x55bd257546b0 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 0x55bd258fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 0x55bd258fa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44524/0 (socket says 172.21.15.82:44524) 2023-07-14T19:36:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 -- 172.21.15.82:0/3314988680 learned_addr learned my addr 172.21.15.82:0/3314988680 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08819bf700 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd257546b0 0x55bd258f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08821c0700 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd25906e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 -- 172.21.15.82:0/3314988680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd257546b0 msgr2=0x55bd258f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd257546b0 0x55bd258f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 -- 172.21.15.82:0/3314988680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 msgr2=0x55bd25906e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd25906e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 -- 172.21.15.82:0/3314988680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd25677d80 con 0x55bd258f9cc0 2023-07-14T19:36:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.855+0000 7f08811be700 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 0x55bd258fa130 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f086c011c60 tx=0x7f086c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4eceffb4b4e7c19 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f0867fff700 1 -- 172.21.15.82:0/3314988680 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00ea40 con 0x55bd258f9cc0 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f0867fff700 1 -- 172.21.15.82:0/3314988680 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086c021070 con 0x55bd258f9cc0 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f0867fff700 1 -- 172.21.15.82:0/3314988680 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c005420 con 0x55bd258f9cc0 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 -- 172.21.15.82:0/3314988680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 msgr2=0x55bd258fa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 0x55bd258fa130 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f086c011c60 tx=0x7f086c014000 comp rx=0 tx=0).stop 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 -- 172.21.15.82:0/3314988680 shutdown_connections 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd25906e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd257546b0 0x55bd258f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/3314988680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd258f9cc0 0x55bd258fa130 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 -- 172.21.15.82:0/3314988680 >> 172.21.15.82:0/3314988680 conn(0x55bd258ee3d0 msgr2=0x55bd258f5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 -- 172.21.15.82:0/3314988680 shutdown_connections 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.856+0000 7f088fdbbb80 1 -- 172.21.15.82:0/3314988680 wait complete. 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 -- start start 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 Processor -- start 2023-07-14T19:36:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd25904430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd258f9cc0 0x55bd25901640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd25904a40 0x55bd25901c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd257e9a60 con 0x55bd257546b0 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd257e9760 con 0x55bd258f9cc0 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd256cd8c0 con 0x55bd25904a40 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f088fdbbb80 10 client.0 _ll_register_callbacks cb 0x55bd257e7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f08819bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd25904430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f08819bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd25904430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44528/0 (socket says 172.21.15.82:44528) 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f08819bf700 1 -- 172.21.15.82:0/2245867166 learned_addr learned my addr 172.21.15.82:0/2245867166 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f08821c0700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd25904a40 0x55bd25901c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.857+0000 7f08811be700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd258f9cc0 0x55bd25901640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08819bf700 1 -- 172.21.15.82:0/2245867166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd25904a40 msgr2=0x55bd25901c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08819bf700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd25904a40 0x55bd25901c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08819bf700 1 -- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd258f9cc0 msgr2=0x55bd25901640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08819bf700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd258f9cc0 0x55bd25901640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08819bf700 1 -- 172.21.15.82:0/2245867166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c017070 con 0x55bd257546b0 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08821c0700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd25904a40 0x55bd25901c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08811be700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd258f9cc0 0x55bd25901640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08819bf700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd25904430 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f087800b6f0 tx=0x7f087809aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16b3ddb89a9e6b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08657fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08657fa700 1 -- 172.21.15.82:0/2245867166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087809a810 con 0x55bd257546b0 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08657fa700 1 -- 172.21.15.82:0/2245867166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f087809f150 con 0x55bd257546b0 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd257a3490 con 0x55bd257546b0 2023-07-14T19:36:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08657fa700 1 -- 172.21.15.82:0/2245867166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08780a34e0 con 0x55bd257546b0 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f088fdbbb80 10 client.16155 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55bd25660c00 con 0x55bd257546b0 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08657fa700 1 -- 172.21.15.82:0/2245867166 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd257a3490 con 0x55bd257546b0 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f08657fa700 1 -- 172.21.15.82:0/2245867166 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f087800fad0 con 0x55bd257546b0 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f088fdbbb80 10 client.16155 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f088fdbbb80 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.858+0000 7f088fdbbb80 20 client.16155 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.859+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25914220 0x55bd25916630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.859+0000 7f088fdbbb80 4 client.16155 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:36:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.859+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd258c7d40 con 0x55bd25914220 2023-07-14T19:36:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.859+0000 7f08811be700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25914220 0x55bd25916630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.859+0000 7f08657fa700 10 client.16155 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.859+0000 7f08811be700 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25914220 0x55bd25916630 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f5684b034eac972 server_cookie=e2b6c2838e613f33 in_seq=0 out_seq=0 2023-07-14T19:36:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f08657fa700 1 -- 172.21.15.82:0/2245867166 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55bd258c7d40 con 0x55bd25914220 2023-07-14T19:36:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f08657fa700 10 client.16155 handle_command_reply: tid=0 2023-07-14T19:36:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 client.16155 shutdown 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 msgr2=0x55bd25904430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd25904430 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f087800b6f0 tx=0x7f087809aa80 comp rx=0 tx=0).stop 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 shutdown_connections 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25914220 0x55bd25916630 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd25904a40 0x55bd25901c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f08657fa700 0 client.16155 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd25904430 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/2245867166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd258f9cc0 0x55bd25901640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 >> 172.21.15.82:0/2245867166 conn(0x55bd258ee3d0 msgr2=0x55bd258f2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 shutdown_connections 2023-07-14T19:36:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 1 -- 172.21.15.82:0/2245867166 wait complete. 2023-07-14T19:36:20.883 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.860+0000 7f088fdbbb80 20 client.16155 trim_cache size 0 max 16384 2023-07-14T19:36:20.887 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.874+0000 7f088fdbbb80 1 Processor -- start 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.874+0000 7f088fdbbb80 1 -- start start 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd258fc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd258f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd258fd4f0 con 0x55bd257546b0 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd25902ad0 con 0x55bd25904a40 2023-07-14T19:36:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd25910e60 con 0x55bd258f9cc0 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd258fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd258fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44536/0 (socket says 172.21.15.82:44536) 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 -- 172.21.15.82:0/4282515475 learned_addr learned my addr 172.21.15.82:0/4282515475 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08811be700 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08821c0700 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd258f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 -- 172.21.15.82:0/4282515475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 msgr2=0x55bd258fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.889 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 -- 172.21.15.82:0/4282515475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 msgr2=0x55bd258f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd258f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 -- 172.21.15.82:0/4282515475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd2569e1f0 con 0x55bd257546b0 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08821c0700 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd258f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f08819bf700 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd258fc880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f087809b0b0 tx=0x7f08780996d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70dc40b75505d60 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f0870daa700 1 -- 172.21.15.82:0/4282515475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087809f150 con 0x55bd257546b0 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f0870daa700 1 -- 172.21.15.82:0/4282515475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08780a34e0 con 0x55bd257546b0 2023-07-14T19:36:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.875+0000 7f0870daa700 1 -- 172.21.15.82:0/4282515475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780a3050 con 0x55bd257546b0 2023-07-14T19:36:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 -- 172.21.15.82:0/4282515475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 msgr2=0x55bd258fc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd258fc880 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f087809b0b0 tx=0x7f08780996d0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 -- 172.21.15.82:0/4282515475 shutdown_connections 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd25904a40 0x55bd258f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd257546b0 0x55bd258fc880 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/4282515475 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 -- 172.21.15.82:0/4282515475 >> 172.21.15.82:0/4282515475 conn(0x55bd258ee3d0 msgr2=0x55bd2590fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 -- 172.21.15.82:0/4282515475 shutdown_connections 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 -- 172.21.15.82:0/4282515475 wait complete. 2023-07-14T19:36:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 -- start start 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.876+0000 7f088fdbbb80 1 Processor -- start 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd257546b0 0x55bd258fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 0x55bd258fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd258fea90 con 0x55bd25904a40 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd258fec00 con 0x55bd257546b0 2023-07-14T19:36:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd258fedc0 con 0x55bd258f9cc0 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f088fdbbb80 10 client.0 _ll_register_callbacks cb 0x55bd257e7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 0x55bd258fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 0x55bd258fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44544/0 (socket says 172.21.15.82:44544) 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 -- 172.21.15.82:0/571956166 learned_addr learned my addr 172.21.15.82:0/571956166 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08811be700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08819bf700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd257546b0 0x55bd258fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 -- 172.21.15.82:0/571956166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 msgr2=0x55bd258fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 -- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd257546b0 msgr2=0x55bd258fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd257546b0 0x55bd258fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.877+0000 7f08821c0700 1 -- 172.21.15.82:0/571956166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd25660c00 con 0x55bd25904a40 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08821c0700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 0x55bd258fe380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f087400e840 tx=0x7f087400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31afbe6d4b3fca6b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08657fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08657fa700 1 -- 172.21.15.82:0/571956166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740189b0 con 0x55bd25904a40 2023-07-14T19:36:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08657fa700 1 -- 172.21.15.82:0/571956166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f0874018b10 con 0x55bd25904a40 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f086c021070 con 0x55bd25904a40 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08657fa700 1 -- 172.21.15.82:0/571956166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0874018cd0 con 0x55bd25904a40 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 10 client.16161 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55bd25676250 con 0x55bd25904a40 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08657fa700 1 -- 172.21.15.82:0/571956166 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086c021070 con 0x55bd25904a40 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08657fa700 1 -- 172.21.15.82:0/571956166 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0874016430 con 0x55bd25904a40 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 20 client.16161 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25968ab0 0x55bd2596aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 4 client.16161 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bd25842e50 con 0x55bd25968ab0 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.878+0000 7f08819bf700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25968ab0 0x55bd2596aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.879+0000 7f08657fa700 10 client.16161 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:20.879+0000 7f08819bf700 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25968ab0 0x55bd2596aec0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a635e5119fcee33d server_cookie=92dc5774b474475c in_seq=0 out_seq=0 2023-07-14T19:36:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f08657fa700 1 -- 172.21.15.82:0/571956166 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bd25842e50 con 0x55bd25968ab0 2023-07-14T19:36:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f08657fa700 10 client.16161 handle_command_reply: tid=0 2023-07-14T19:36:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 client.16161 shutdown 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 msgr2=0x55bd258fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 0x55bd258fe380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f087400e840 tx=0x7f087400eb50 comp rx=0 tx=0).stop 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 shutdown_connections 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd25904a40 0x55bd258fe380 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bd257546b0 0x55bd258fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55bd25968ab0 0x55bd2596aec0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 --2- 172.21.15.82:0/571956166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55bd258f9cc0 0x55bd258fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f08657fa700 0 client.16161 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 >> 172.21.15.82:0/571956166 conn(0x55bd258ee3d0 msgr2=0x55bd258f59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 shutdown_connections 2023-07-14T19:36:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 1 -- 172.21.15.82:0/571956166 wait complete. 2023-07-14T19:36:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.126+0000 7f088fdbbb80 20 client.16161 trim_cache size 0 max 16384 2023-07-14T19:36:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:36:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:36:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:36:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f08680a12c0 msgr2=0x7f08680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f08680a12c0 0x7f08680a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f087c196490 tx=0x7f086c009260 comp rx=0 tx=0).stop 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c0ff2e0 msgr2=0x7f087c194cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c0ff2e0 0x7f087c194cb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878009580 comp rx=0 tx=0).stop 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 shutdown_connections 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f08680a12c0 0x7f08680a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f087c105490 0x7f087c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f087c0ff2e0 0x7f087c194cb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 --2- 172.21.15.82:0/1397311883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f087c104b80 0x7f087c1951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 >> 172.21.15.82:0/1397311883 conn(0x7f087c076ae0 msgr2=0x7f087c102820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 shutdown_connections 2023-07-14T19:36:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:21.127+0000 7f0883c23700 1 -- 172.21.15.82:0/1397311883 wait complete. 2023-07-14T19:36:21.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:21 smithi133 ceph-mon[128894]: pgmap v242: 97 pgs: 15 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 75 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2023-07-14T19:36:21.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:21 smithi082 ceph-mon[127461]: pgmap v242: 97 pgs: 15 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 75 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2023-07-14T19:36:21.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:21 smithi161 ceph-mon[129184]: pgmap v242: 97 pgs: 15 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 75 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 352 op/s 2023-07-14T19:36:23.444 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:23 smithi133 ceph-mon[128894]: pgmap v243: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2023-07-14T19:36:23.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:23 smithi133 ceph-mon[128894]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T19:36:23.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:23 smithi082 ceph-mon[127461]: pgmap v243: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2023-07-14T19:36:23.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:23 smithi082 ceph-mon[127461]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T19:36:23.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:23 smithi161 ceph-mon[129184]: pgmap v243: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2023-07-14T19:36:23.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:23 smithi161 ceph-mon[129184]: osdmap e91: 12 total, 12 up, 12 in 2023-07-14T19:36:25.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:25 smithi133 ceph-mon[128894]: pgmap v245: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2023-07-14T19:36:25.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:25 smithi082 ceph-mon[127461]: pgmap v245: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2023-07-14T19:36:25.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:25 smithi161 ceph-mon[129184]: pgmap v245: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 305 op/s 2023-07-14T19:36:27.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:27 smithi133 ceph-mon[128894]: pgmap v246: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2023-07-14T19:36:27.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:27 smithi082 ceph-mon[127461]: pgmap v246: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2023-07-14T19:36:27.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:27 smithi161 ceph-mon[129184]: pgmap v246: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2023-07-14T19:36:29.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:29 smithi133 ceph-mon[128894]: pgmap v247: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2023-07-14T19:36:29.475 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:29 smithi082 ceph-mon[127461]: pgmap v247: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2023-07-14T19:36:29.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:29 smithi161 ceph-mon[129184]: pgmap v247: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2023-07-14T19:36:31.449 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:31 smithi133 ceph-mon[128894]: pgmap v248: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2023-07-14T19:36:31.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:31 smithi082 ceph-mon[127461]: pgmap v248: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2023-07-14T19:36:31.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:31 smithi161 ceph-mon[129184]: pgmap v248: 97 pgs: 97 active+clean; 1.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2023-07-14T19:36:33.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:33 smithi133 ceph-mon[128894]: pgmap v249: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2023-07-14T19:36:33.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:33 smithi082 ceph-mon[127461]: pgmap v249: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2023-07-14T19:36:33.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:33 smithi161 ceph-mon[129184]: pgmap v249: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 346 op/s 2023-07-14T19:36:34.266 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 -- 172.21.15.82:0/1406006667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e4104f90 msgr2=0x7f89e410caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 --2- 172.21.15.82:0/1406006667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e4104f90 0x7f89e410caf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00b0a0 tx=0x7f89dc00b3b0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 -- 172.21.15.82:0/1406006667 shutdown_connections 2023-07-14T19:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 --2- 172.21.15.82:0/1406006667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e4104f90 0x7f89e410caf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 --2- 172.21.15.82:0/1406006667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89e40fe3f0 0x7f89e40fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 --2- 172.21.15.82:0/1406006667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89e40fed90 0x7f89e4104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 -- 172.21.15.82:0/1406006667 >> 172.21.15.82:0/1406006667 conn(0x7f89e40760c0 msgr2=0x7f89e40764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 -- 172.21.15.82:0/1406006667 shutdown_connections 2023-07-14T19:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 -- 172.21.15.82:0/1406006667 wait complete. 2023-07-14T19:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 Processor -- start 2023-07-14T19:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.253+0000 7f89eaa77700 1 -- start start 2023-07-14T19:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89e40fe3f0 0x7f89e4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 0x7f89e4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89e4104f90 0x7f89e419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e419d1d0 con 0x7f89e40fed90 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e419d340 con 0x7f89e40fe3f0 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e419d500 con 0x7f89e4104f90 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 0x7f89e4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 0x7f89e4198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44556/0 (socket says 172.21.15.82:44556) 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 -- 172.21.15.82:0/648412135 learned_addr learned my addr 172.21.15.82:0/648412135 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e8813700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89e40fe3f0 0x7f89e4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e9014700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89e4104f90 0x7f89e419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 -- 172.21.15.82:0/648412135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89e4104f90 msgr2=0x7f89e419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89e4104f90 0x7f89e419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 -- 172.21.15.82:0/648412135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89e40fe3f0 msgr2=0x7f89e4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89e40fe3f0 0x7f89e4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 -- 172.21.15.82:0/648412135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc009d00 con 0x7f89e40fed90 2023-07-14T19:36:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e3fff700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 0x7f89e4198520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f89d000e970 tx=0x7f89d000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e1ffb700 1 -- 172.21.15.82:0/648412135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0009790 con 0x7f89e40fed90 2023-07-14T19:36:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e419d7e0 con 0x7f89e40fed90 2023-07-14T19:36:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e1ffb700 1 -- 172.21.15.82:0/648412135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89d000c7e0 con 0x7f89e40fed90 2023-07-14T19:36:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.254+0000 7f89e1ffb700 1 -- 172.21.15.82:0/648412135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d00054e0 con 0x7f89e40fed90 2023-07-14T19:36:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.255+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e41a5020 con 0x7f89e40fed90 2023-07-14T19:36:34.273 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.256+0000 7f89e1ffb700 1 -- 172.21.15.82:0/648412135 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f89d000c950 con 0x7f89e40fed90 2023-07-14T19:36:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.257+0000 7f89e1ffb700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f89d40a12c0 0x7f89d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.257+0000 7f89e1ffb700 1 -- 172.21.15.82:0/648412135 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f89d0014070 con 0x7f89e40fed90 2023-07-14T19:36:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.257+0000 7f89e8813700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f89d40a12c0 0x7f89d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.257+0000 7f89e8813700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f89d40a12c0 0x7f89d40a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009c60 tx=0x7f89d8009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:34.278 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 Processor -- start 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 -- start start 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a07d96b0 0x5625a0983550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a097d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a097df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a0882c50 con 0x5625a0983f00 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a08d59e0 con 0x5625a0983a90 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a08cee70 con 0x5625a07d96b0 2023-07-14T19:36:34.279 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89e9014700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a097df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89e9014700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a097df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44566/0 (socket says 172.21.15.82:44566) 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89e9014700 1 -- 172.21.15.82:0/4281703026 learned_addr learned my addr 172.21.15.82:0/4281703026 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89e3fff700 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a097d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.258+0000 7f89e8813700 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a07d96b0 0x5625a0983550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e9014700 1 -- 172.21.15.82:0/4281703026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a07d96b0 msgr2=0x5625a0983550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e9014700 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a07d96b0 0x5625a0983550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e9014700 1 -- 172.21.15.82:0/4281703026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 msgr2=0x5625a097d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e9014700 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a097d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e9014700 1 -- 172.21.15.82:0/4281703026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625a06fcd80 con 0x5625a0983f00 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e3fff700 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a097d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89e9014700 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a097df80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009c70 tx=0x7f89dc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe68ac529a9a976 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89ceffd700 1 -- 172.21.15.82:0/4281703026 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00e040 con 0x5625a0983f00 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89ceffd700 1 -- 172.21.15.82:0/4281703026 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89dc003a00 con 0x5625a0983f00 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.259+0000 7f89ceffd700 1 -- 172.21.15.82:0/4281703026 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc01c6a0 con 0x5625a0983f00 2023-07-14T19:36:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/4281703026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 msgr2=0x5625a097df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a097df80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009c70 tx=0x7f89dc0092e0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/4281703026 shutdown_connections 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a097df80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a07d96b0 0x5625a0983550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/4281703026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a097d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/4281703026 >> 172.21.15.82:0/4281703026 conn(0x5625a09733d0 msgr2=0x5625a0978530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/4281703026 shutdown_connections 2023-07-14T19:36:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/4281703026 wait complete. 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 -- start start 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.260+0000 7f89f6c0fb80 1 Processor -- start 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 0x5625a098cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a098d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983f00 0x5625a0986f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a086ea60 con 0x5625a07d96b0 2023-07-14T19:36:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a086e760 con 0x5625a0983a90 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a07528c0 con 0x5625a0983f00 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 0x5625a098cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 10 client.0 _ll_register_callbacks cb 0x5625a086c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 0x5625a098cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44568/0 (socket says 172.21.15.82:44568) 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 -- 172.21.15.82:0/685492810 learned_addr learned my addr 172.21.15.82:0/685492810 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e3fff700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a098d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e9014700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983f00 0x5625a0986f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 -- 172.21.15.82:0/685492810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983f00 msgr2=0x5625a0986f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983f00 0x5625a0986f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 -- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 msgr2=0x5625a098d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a098d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 -- 172.21.15.82:0/685492810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e41a5020 con 0x5625a07d96b0 2023-07-14T19:36:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89e8813700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 0x5625a098cd00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f89d8003900 tx=0x7f89d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca9eac25c42a56 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89c3fff700 1 -- 172.21.15.82:0/685492810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8022070 con 0x5625a07d96b0 2023-07-14T19:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89c3fff700 1 -- 172.21.15.82:0/685492810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f89d8027420 con 0x5625a07d96b0 2023-07-14T19:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625a0828490 con 0x5625a07d96b0 2023-07-14T19:36:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.261+0000 7f89c3fff700 1 -- 172.21.15.82:0/685492810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d80275e0 con 0x5625a07d96b0 2023-07-14T19:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 10 client.16173 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f89e410d2e0 con 0x5625a07d96b0 2023-07-14T19:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89c3fff700 1 -- 172.21.15.82:0/685492810 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5625a0828490 con 0x5625a07d96b0 2023-07-14T19:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89c3fff700 1 -- 172.21.15.82:0/685492810 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f89d802e020 con 0x5625a07d96b0 2023-07-14T19:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 20 client.16173 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09971f0 0x5625a0999600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 4 client.16173 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625a094cd40 con 0x5625a09971f0 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89e3fff700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09971f0 0x5625a0999600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89c3fff700 10 client.16173 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.262+0000 7f89e3fff700 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09971f0 0x5625a0999600 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c621ce56925f443a server_cookie=bd4ac3a379bc071e in_seq=0 out_seq=0 2023-07-14T19:36:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89c3fff700 1 -- 172.21.15.82:0/685492810 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5625a094cd40 con 0x5625a09971f0 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89c3fff700 10 client.16173 handle_command_reply: tid=0 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 client.16173 shutdown 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 msgr2=0x5625a098cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 0x5625a098cd00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f89d8003900 tx=0x7f89d80047c0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 shutdown_connections 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09971f0 0x5625a0999600 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983f00 0x5625a0986f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a07d96b0 0x5625a098cd00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89c3fff700 0 client.16173 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/685492810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a0983a90 0x5625a098d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 >> 172.21.15.82:0/685492810 conn(0x5625a09733d0 msgr2=0x5625a0977c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 shutdown_connections 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/685492810 wait complete. 2023-07-14T19:36:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.263+0000 7f89f6c0fb80 20 client.16173 trim_cache size 0 max 16384 2023-07-14T19:36:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.278+0000 7f89f6c0fb80 1 Processor -- start 2023-07-14T19:36:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.278+0000 7f89f6c0fb80 1 -- start start 2023-07-14T19:36:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a0987080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a0984a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a09783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a099e410 con 0x5625a0983f00 2023-07-14T19:36:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a0984f80 con 0x5625a07d96b0 2023-07-14T19:36:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a09797a0 con 0x5625a0983a90 2023-07-14T19:36:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e9014700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a09783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e9014700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a09783b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44580/0 (socket says 172.21.15.82:44580) 2023-07-14T19:36:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a0987080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e9014700 1 -- 172.21.15.82:0/2209324232 learned_addr learned my addr 172.21.15.82:0/2209324232 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e3fff700 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a0984a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 -- 172.21.15.82:0/2209324232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 msgr2=0x5625a0984a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a0984a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 -- 172.21.15.82:0/2209324232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 msgr2=0x5625a09783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a09783b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 -- 172.21.15.82:0/2209324232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625a06e5c00 con 0x5625a07d96b0 2023-07-14T19:36:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e9014700 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a09783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89e8813700 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a0987080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89d80309e0 tx=0x7f89d8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488494a0ce653b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89cf7fe700 1 -- 172.21.15.82:0/2209324232 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8022040 con 0x5625a07d96b0 2023-07-14T19:36:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89cf7fe700 1 -- 172.21.15.82:0/2209324232 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89d80045e0 con 0x5625a07d96b0 2023-07-14T19:36:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.279+0000 7f89cf7fe700 1 -- 172.21.15.82:0/2209324232 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8039de0 con 0x5625a07d96b0 2023-07-14T19:36:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/2209324232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 msgr2=0x5625a0987080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a0987080 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89d80309e0 tx=0x7f89d8011ab0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/2209324232 shutdown_connections 2023-07-14T19:36:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a09783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a0987080 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/2209324232 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a0984a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/2209324232 >> 172.21.15.82:0/2209324232 conn(0x5625a09733d0 msgr2=0x5625a0984300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/2209324232 shutdown_connections 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/2209324232 wait complete. 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 -- start start 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.280+0000 7f89f6c0fb80 1 Processor -- start 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a098cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a098d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a098af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a098b680 con 0x5625a0983f00 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a098b7f0 con 0x5625a07d96b0 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625a098b9b0 con 0x5625a0983a90 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89f6c0fb80 10 client.0 _ll_register_callbacks cb 0x5625a086c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a098af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e8813700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a098cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e8813700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a098cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:40930/0 (socket says 172.21.15.82:40930) 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a098af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44586/0 (socket says 172.21.15.82:44586) 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e8813700 1 -- 172.21.15.82:0/3430822276 learned_addr learned my addr 172.21.15.82:0/3430822276 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e3fff700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a098d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 -- 172.21.15.82:0/3430822276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 msgr2=0x5625a098d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a098d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 -- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 msgr2=0x5625a098cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a098cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 -- 172.21.15.82:0/3430822276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625a06fcd80 con 0x5625a0983f00 2023-07-14T19:36:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e8813700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a098cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89e9014700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a098af60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f89dc000f80 tx=0x7f89dc003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc4f2d543349420 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:36:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.281+0000 7f89c3fff700 1 -- 172.21.15.82:0/3430822276 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc029070 con 0x5625a0983f00 2023-07-14T19:36:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89c3fff700 1 -- 172.21.15.82:0/3430822276 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f89dc0060d0 con 0x5625a0983f00 2023-07-14T19:36:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89dc003a00 con 0x5625a0983f00 2023-07-14T19:36:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89c3fff700 1 -- 172.21.15.82:0/3430822276 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89dc00e040 con 0x5625a0983f00 2023-07-14T19:36:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 10 client.16179 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f89e410d2e0 con 0x5625a0983f00 2023-07-14T19:36:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89c3fff700 1 -- 172.21.15.82:0/3430822276 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89dc003a00 con 0x5625a0983f00 2023-07-14T19:36:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89c3fff700 1 -- 172.21.15.82:0/3430822276 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f89dc019030 con 0x5625a0983f00 2023-07-14T19:36:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 20 client.16179 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09ebbf0 0x5625a09ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 4 client.16179 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:36:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5625a08c7e50 con 0x5625a09ebbf0 2023-07-14T19:36:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.282+0000 7f89e8813700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09ebbf0 0x5625a09ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.283+0000 7f89e8813700 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09ebbf0 0x5625a09ee000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779fc15b542179e6 server_cookie=2b98963a2efbcac8 in_seq=0 out_seq=0 2023-07-14T19:36:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.283+0000 7f89c3fff700 10 client.16179 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89c3fff700 1 -- 172.21.15.82:0/3430822276 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5625a08c7e50 con 0x5625a09ebbf0 2023-07-14T19:36:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89c3fff700 10 client.16179 handle_command_reply: tid=0 2023-07-14T19:36:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 client.16179 shutdown 2023-07-14T19:36:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 msgr2=0x5625a098af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a098af60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f89dc000f80 tx=0x7f89dc003da0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 shutdown_connections 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5625a0983f00 0x5625a098af60 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625a07d96b0 0x5625a098cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5625a09ebbf0 0x5625a09ee000 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 --2- 172.21.15.82:0/3430822276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5625a0983a90 0x5625a098d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89c3fff700 0 client.16179 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 >> 172.21.15.82:0/3430822276 conn(0x5625a09733d0 msgr2=0x5625a09828c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 shutdown_connections 2023-07-14T19:36:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 1 -- 172.21.15.82:0/3430822276 wait complete. 2023-07-14T19:36:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.316+0000 7f89f6c0fb80 20 client.16179 trim_cache size 0 max 16384 2023-07-14T19:36:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:36:34.332 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:36:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:36:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:36:34.333 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f89d40a12c0 msgr2=0x7f89d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f89d40a12c0 0x7f89d40a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009c60 tx=0x7f89d8009260 comp rx=0 tx=0).stop 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 msgr2=0x7f89e4198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 0x7f89e4198520 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f89d000e970 tx=0x7f89d000ec80 comp rx=0 tx=0).stop 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 shutdown_connections 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f89d40a12c0 0x7f89d40a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89e4104f90 0x7f89e419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89e40fe3f0 0x7f89e4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 --2- 172.21.15.82:0/648412135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40fed90 0x7f89e4198520 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 >> 172.21.15.82:0/648412135 conn(0x7f89e40760c0 msgr2=0x7f89e40fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 shutdown_connections 2023-07-14T19:36:34.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:34.317+0000 7f89eaa77700 1 -- 172.21.15.82:0/648412135 wait complete. 2023-07-14T19:36:35.445 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:35 smithi133 ceph-mon[128894]: pgmap v250: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2023-07-14T19:36:35.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:35 smithi082 ceph-mon[127461]: pgmap v250: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2023-07-14T19:36:35.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:35 smithi161 ceph-mon[129184]: pgmap v250: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2023-07-14T19:36:37.448 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:37 smithi133 ceph-mon[128894]: pgmap v251: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2023-07-14T19:36:37.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:37 smithi082 ceph-mon[127461]: pgmap v251: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2023-07-14T19:36:37.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:37 smithi161 ceph-mon[129184]: pgmap v251: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2023-07-14T19:36:39.448 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:39 smithi133 ceph-mon[128894]: pgmap v252: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2023-07-14T19:36:39.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:39 smithi082 ceph-mon[127461]: pgmap v252: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2023-07-14T19:36:39.505 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:39 smithi161 ceph-mon[129184]: pgmap v252: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2023-07-14T19:36:41.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:41 smithi133 ceph-mon[128894]: pgmap v253: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 329 op/s 2023-07-14T19:36:41.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:41 smithi082 ceph-mon[127461]: pgmap v253: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 329 op/s 2023-07-14T19:36:41.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:41 smithi161 ceph-mon[129184]: pgmap v253: 97 pgs: 97 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 329 op/s 2023-07-14T19:36:43.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:43 smithi133 ceph-mon[128894]: pgmap v254: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 340 op/s 2023-07-14T19:36:43.476 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:43 smithi082 ceph-mon[127461]: pgmap v254: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 340 op/s 2023-07-14T19:36:43.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:43 smithi161 ceph-mon[129184]: pgmap v254: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 340 op/s 2023-07-14T19:36:45.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:45 smithi133 ceph-mon[128894]: pgmap v255: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2023-07-14T19:36:45.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:45 smithi082 ceph-mon[127461]: pgmap v255: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2023-07-14T19:36:45.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:45 smithi161 ceph-mon[129184]: pgmap v255: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2023-07-14T19:36:47.447 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:47 smithi133 ceph-mon[128894]: pgmap v256: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2023-07-14T19:36:47.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:47 smithi082 ceph-mon[127461]: pgmap v256: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2023-07-14T19:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 -- 172.21.15.82:0/2887417437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a80ffeb0 msgr2=0x7f04a8100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 --2- 172.21.15.82:0/2887417437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a80ffeb0 0x7f04a8100280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f04980099c0 tx=0x7f0498009cd0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 -- 172.21.15.82:0/2887417437 shutdown_connections 2023-07-14T19:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 --2- 172.21.15.82:0/2887417437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04a8104ef0 0x7f04a810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.485 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 --2- 172.21.15.82:0/2887417437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a80ffeb0 0x7f04a8100280 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 --2- 172.21.15.82:0/2887417437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8100850 0x7f04a8104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 -- 172.21.15.82:0/2887417437 >> 172.21.15.82:0/2887417437 conn(0x7f04a80fb780 msgr2=0x7f04a80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.470+0000 7f04add6e700 1 -- 172.21.15.82:0/2887417437 shutdown_connections 2023-07-14T19:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.471+0000 7f04add6e700 1 -- 172.21.15.82:0/2887417437 wait complete. 2023-07-14T19:36:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.471+0000 7f04add6e700 1 Processor -- start 2023-07-14T19:36:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 -- start start 2023-07-14T19:36:47.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04a80ffeb0 0x7f04a8195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 0x7f04a81962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8104ef0 0x7f04a819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a819af90 con 0x7f04a8100850 2023-07-14T19:36:47.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a819b100 con 0x7f04a8104ef0 2023-07-14T19:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04add6e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a819b2c0 con 0x7f04a80ffeb0 2023-07-14T19:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.473+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 0x7f04a81962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 0x7f04a81962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44598/0 (socket says 172.21.15.82:44598) 2023-07-14T19:36:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 -- 172.21.15.82:0/515157348 learned_addr learned my addr 172.21.15.82:0/515157348 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a7fff700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8104ef0 0x7f04a819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 -- 172.21.15.82:0/515157348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04a80ffeb0 msgr2=0x7f04a8195da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T19:36:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04a80ffeb0 0x7f04a8195da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 -- 172.21.15.82:0/515157348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8104ef0 msgr2=0x7f04a819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8104ef0 0x7f04a819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 -- 172.21.15.82:0/515157348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980095f0 con 0x7f04a8100850 2023-07-14T19:36:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a7fff700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8104ef0 0x7f04a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:36:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 0x7f04a81962e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e940 tx=0x7f049c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a4ff9700 1 -- 172.21.15.82:0/515157348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c009730 con 0x7f04a8100850 2023-07-14T19:36:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a819b5a0 con 0x7f04a8100850 2023-07-14T19:36:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a4ff9700 1 -- 172.21.15.82:0/515157348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049c00cbd0 con 0x7f04a8100850 2023-07-14T19:36:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04a4ff9700 1 -- 172.21.15.82:0/515157348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c018680 con 0x7f04a8100850 2023-07-14T19:36:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.474+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a81a2de0 con 0x7f04a8100850 2023-07-14T19:36:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.478+0000 7f04a4ff9700 1 -- 172.21.15.82:0/515157348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f049c00cd40 con 0x7f04a8100850 2023-07-14T19:36:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.479+0000 7f04a4ff9700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f04900a1340 0x7f04900a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.479+0000 7f04a4ff9700 1 -- 172.21.15.82:0/515157348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f049c014070 con 0x7f04a8100850 2023-07-14T19:36:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.479+0000 7f04a77fe700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f04900a1340 0x7f04900a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 Processor -- start 2023-07-14T19:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 -- start start 2023-07-14T19:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04a77fe700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f04900a1340 0x7f04900a37f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04980058e0 tx=0x7f049800b540 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eeb50 0x563f250e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eefc0 0x563f250e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f24fedc50 con 0x563f24f446b0 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f250409e0 con 0x563f250eeb50 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f25039e70 con 0x563f250eefc0 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04a77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04a77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250ee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44604/0 (socket says 172.21.15.82:44604) 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04a77fe700 1 -- 172.21.15.82:0/104087910 learned_addr learned my addr 172.21.15.82:0/104087910 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eeb50 0x563f250e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.480+0000 7f04a7fff700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eefc0 0x563f250e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a77fe700 1 -- 172.21.15.82:0/104087910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eefc0 msgr2=0x563f250e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a77fe700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eefc0 0x563f250e8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a77fe700 1 -- 172.21.15.82:0/104087910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eeb50 msgr2=0x563f250e89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a77fe700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eeb50 0x563f250e89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a77fe700 1 -- 172.21.15.82:0/104087910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f24e67d80 con 0x563f24f446b0 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a7fff700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eefc0 0x563f250e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:36:47.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eeb50 0x563f250e89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:36:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f04a77fe700 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250ee610 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f04980099c0 tx=0x7f0498009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f8fbf856e95993 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f048dffb700 1 -- 172.21.15.82:0/104087910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498028070 con 0x563f24f446b0 2023-07-14T19:36:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f048dffb700 1 -- 172.21.15.82:0/104087910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0498005a00 con 0x563f24f446b0 2023-07-14T19:36:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.481+0000 7f048dffb700 1 -- 172.21.15.82:0/104087910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049801c070 con 0x563f24f446b0 2023-07-14T19:36:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.482+0000 7f04b9f06b80 1 -- 172.21.15.82:0/104087910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 msgr2=0x563f250ee610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.482+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250ee610 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f04980099c0 tx=0x7f0498009310 comp rx=0 tx=0).stop 2023-07-14T19:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 -- 172.21.15.82:0/104087910 shutdown_connections 2023-07-14T19:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eefc0 0x563f250e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250ee610 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/104087910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eeb50 0x563f250e89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 -- 172.21.15.82:0/104087910 >> 172.21.15.82:0/104087910 conn(0x563f250de3d0 msgr2=0x563f250e3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:47.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 -- 172.21.15.82:0/104087910 shutdown_connections 2023-07-14T19:36:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 -- 172.21.15.82:0/104087910 wait complete. 2023-07-14T19:36:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.483+0000 7f04b9f06b80 1 -- start start 2023-07-14T19:36:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 Processor -- start 2023-07-14T19:36:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 0x563f250f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f24fd9a60 con 0x563f24f446b0 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f24fd9760 con 0x563f250eefc0 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f24ebd8c0 con 0x563f250eeb50 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04b9f06b80 10 client.0 _ll_register_callbacks cb 0x563f24fd7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 0x563f250f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 0x563f250f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53878/0 (socket says 172.21.15.82:53878) 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.484+0000 7f04a6ffd700 1 -- 172.21.15.82:0/464080665 learned_addr learned my addr 172.21.15.82:0/464080665 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a7fff700 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a6ffd700 1 -- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 msgr2=0x563f250f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a6ffd700 1 -- 172.21.15.82:0/464080665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 msgr2=0x563f250f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a6ffd700 1 -- 172.21.15.82:0/464080665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a81a2de0 con 0x563f250eeb50 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 0x563f250f7280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e910 tx=0x7f049c09e0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97b1d3c5d8fd88f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f0486ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f0486ffd700 1 -- 172.21.15.82:0/464080665 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0a2030 con 0x563f250eeb50 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f0486ffd700 1 -- 172.21.15.82:0/464080665 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f049c0a2190 con 0x563f250eeb50 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f24f93490 con 0x563f250eeb50 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.485+0000 7f0486ffd700 1 -- 172.21.15.82:0/464080665 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f049c0b82b0 con 0x563f250eeb50 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.487+0000 7f04b9f06b80 10 client.25741 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.487+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f04a810d300 con 0x563f250eeb50 2023-07-14T19:36:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f0486ffd700 1 -- 172.21.15.82:0/464080665 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049c0c8020 con 0x563f250eeb50 2023-07-14T19:36:47.508 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:47 smithi161 ceph-mon[129184]: pgmap v256: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 318 op/s 2023-07-14T19:36:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f0486ffd700 1 -- 172.21.15.82:0/464080665 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f049c0b2a50 con 0x563f250eeb50 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04b9f06b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04b9f06b80 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04b9f06b80 20 client.25741 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25102310 0x563f25104720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04b9f06b80 4 client.25741 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f250b7d40 con 0x563f25102310 2023-07-14T19:36:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.488+0000 7f04a77fe700 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25102310 0x563f25104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.491+0000 7f0486ffd700 10 client.25741 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.491+0000 7f04a77fe700 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25102310 0x563f25104720 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec75055c3b956546 server_cookie=de64fbae75fe808d in_seq=0 out_seq=0 2023-07-14T19:36:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f0486ffd700 1 -- 172.21.15.82:0/464080665 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563f250b7d40 con 0x563f25102310 2023-07-14T19:36:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f0486ffd700 10 client.25741 handle_command_reply: tid=0 2023-07-14T19:36:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 client.25741 shutdown 2023-07-14T19:36:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 msgr2=0x563f250f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 0x563f250f7280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e910 tx=0x7f049c09e0a0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 shutdown_connections 2023-07-14T19:36:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f24f446b0 0x563f250f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25102310 0x563f25104720 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/464080665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f250eeb50 0x563f250f7280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f0486ffd700 0 client.25741 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 >> 172.21.15.82:0/464080665 conn(0x563f250de3d0 msgr2=0x563f250e3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 shutdown_connections 2023-07-14T19:36:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.611+0000 7f04b9f06b80 1 -- 172.21.15.82:0/464080665 wait complete. 2023-07-14T19:36:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.612+0000 7f04b9f06b80 20 client.25741 trim_cache size 0 max 16384 2023-07-14T19:36:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 Processor -- start 2023-07-14T19:36:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 -- start start 2023-07-14T19:36:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f250ff8f0 con 0x563f250eeb50 2023-07-14T19:36:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f250f35f0 con 0x563f250eefc0 2023-07-14T19:36:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f250e0450 con 0x563f24f446b0 2023-07-14T19:36:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44620/0 (socket says 172.21.15.82:44620) 2023-07-14T19:36:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 -- 172.21.15.82:0/1912548164 learned_addr learned my addr 172.21.15.82:0/1912548164 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a7fff700 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a77fe700 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 -- 172.21.15.82:0/1912548164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 msgr2=0x563f250fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 -- 172.21.15.82:0/1912548164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 msgr2=0x563f250ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250ec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 -- 172.21.15.82:0/1912548164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f24e50c00 con 0x563f250eeb50 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a77fe700 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.627+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f2400 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f049c09a000 tx=0x7f049c0a2640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8840360d3e3a9e0f server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f048e7fc700 1 -- 172.21.15.82:0/1912548164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0b2ea0 con 0x563f250eeb50 2023-07-14T19:36:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f048e7fc700 1 -- 172.21.15.82:0/1912548164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049c0c8070 con 0x563f250eeb50 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f048e7fc700 1 -- 172.21.15.82:0/1912548164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0960c0 con 0x563f250eeb50 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 -- 172.21.15.82:0/1912548164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 msgr2=0x563f250f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f2400 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f049c09a000 tx=0x7f049c0a2640 comp rx=0 tx=0).stop 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 -- 172.21.15.82:0/1912548164 shutdown_connections 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/1912548164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f2400 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 -- 172.21.15.82:0/1912548164 >> 172.21.15.82:0/1912548164 conn(0x563f250de3d0 msgr2=0x563f250e79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 -- 172.21.15.82:0/1912548164 shutdown_connections 2023-07-14T19:36:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.628+0000 7f04b9f06b80 1 -- 172.21.15.82:0/1912548164 wait complete. 2023-07-14T19:36:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 -- start start 2023-07-14T19:36:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 Processor -- start 2023-07-14T19:36:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f250f8490 con 0x563f250eeb50 2023-07-14T19:36:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f250f8600 con 0x563f250eefc0 2023-07-14T19:36:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f25177550 con 0x563f24f446b0 2023-07-14T19:36:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04b9f06b80 10 client.0 _ll_register_callbacks cb 0x7f049c0999d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:36:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44626/0 (socket says 172.21.15.82:44626) 2023-07-14T19:36:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04a6ffd700 1 -- 172.21.15.82:0/2922898562 learned_addr learned my addr 172.21.15.82:0/2922898562 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:36:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.629+0000 7f04a7fff700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a77fe700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a6ffd700 1 -- 172.21.15.82:0/2922898562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 msgr2=0x563f250fba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a6ffd700 1 -- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 msgr2=0x563f250f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:36:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a6ffd700 1 -- 172.21.15.82:0/2922898562 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a810d300 con 0x563f250eeb50 2023-07-14T19:36:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04a6ffd700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f7860 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f049c009230 tx=0x7f049c0b84e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ca197d619dda0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:36:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f0486ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:36:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f0486ffd700 1 -- 172.21.15.82:0/2922898562 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c0ca4b0 con 0x563f250eeb50 2023-07-14T19:36:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f0486ffd700 1 -- 172.21.15.82:0/2922898562 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f049c0c9080 con 0x563f250eeb50 2023-07-14T19:36:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0498005a00 con 0x563f250eeb50 2023-07-14T19:36:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f0486ffd700 1 -- 172.21.15.82:0/2922898562 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f049c0ca610 con 0x563f250eeb50 2023-07-14T19:36:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 10 client.16194 fetch_fsmap learned FSMap version 44 2023-07-14T19:36:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563f24e8e1f0 con 0x563f250eeb50 2023-07-14T19:36:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f0486ffd700 1 -- 172.21.15.82:0/2922898562 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0498005a00 con 0x563f250eeb50 2023-07-14T19:36:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f0486ffd700 1 -- 172.21.15.82:0/2922898562 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f049c07f070 con 0x563f250eeb50 2023-07-14T19:36:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:36:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:36:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 20 client.16194 populate_metadata read hostname 'smithi082' 2023-07-14T19:36:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25178710 0x563f2517ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:36:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 4 client.16194 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:36:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.630+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563f25032e50 con 0x563f25178710 2023-07-14T19:36:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.631+0000 7f04a77fe700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25178710 0x563f2517ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:36:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.631+0000 7f0486ffd700 10 client.16194 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.631+0000 7f04a77fe700 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25178710 0x563f2517ab20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2ca44888fa4b9c server_cookie=e18b36b36f3ee0a in_seq=0 out_seq=0 2023-07-14T19:36:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f0486ffd700 1 -- 172.21.15.82:0/2922898562 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563f25032e50 con 0x563f25178710 2023-07-14T19:36:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f0486ffd700 10 client.16194 handle_command_reply: tid=0 2023-07-14T19:36:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 client.16194 shutdown 2023-07-14T19:36:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 msgr2=0x563f250f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f7860 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f049c009230 tx=0x7f049c0b84e0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 shutdown_connections 2023-07-14T19:36:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f250eefc0 0x563f250f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.863 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563f25178710 0x563f2517ab20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563f24f446b0 0x563f250fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 --2- 172.21.15.82:0/2922898562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f250eeb50 0x563f250f7860 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 >> 172.21.15.82:0/2922898562 conn(0x563f250de3d0 msgr2=0x563f250e79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f0486ffd700 0 client.16194 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:36:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 shutdown_connections 2023-07-14T19:36:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.847+0000 7f04b9f06b80 1 -- 172.21.15.82:0/2922898562 wait complete. 2023-07-14T19:36:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.848+0000 7f04b9f06b80 20 client.16194 trim_cache size 0 max 16384 2023-07-14T19:36:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:36:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:36:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:36:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:36:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.849+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f04900a1340 msgr2=0x7f04900a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.849+0000 7f04add6e700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f04900a1340 0x7f04900a37f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04980058e0 tx=0x7f049800b540 comp rx=0 tx=0).stop 2023-07-14T19:36:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.849+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 msgr2=0x7f04a81962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:36:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.849+0000 7f04add6e700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 0x7f04a81962e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f049c00e940 tx=0x7f049c00ed00 comp rx=0 tx=0).stop 2023-07-14T19:36:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 shutdown_connections 2023-07-14T19:36:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f04900a1340 0x7f04900a37f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04a8104ef0 0x7f04a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04a80ffeb0 0x7f04a8195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 --2- 172.21.15.82:0/515157348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a8100850 0x7f04a81962e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:36:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 >> 172.21.15.82:0/515157348 conn(0x7f04a80fb780 msgr2=0x7f04a80fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:36:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 shutdown_connections 2023-07-14T19:36:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:36:47.850+0000 7f04add6e700 1 -- 172.21.15.82:0/515157348 wait complete. 2023-07-14T19:36:49.447 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:49 smithi133 ceph-mon[128894]: pgmap v257: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2023-07-14T19:36:49.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:49 smithi082 ceph-mon[127461]: pgmap v257: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2023-07-14T19:36:49.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:49 smithi161 ceph-mon[129184]: pgmap v257: 97 pgs: 97 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2023-07-14T19:36:51.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:51 smithi133 ceph-mon[128894]: pgmap v258: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-07-14T19:36:51.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:51 smithi082 ceph-mon[127461]: pgmap v258: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-07-14T19:36:51.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:51 smithi161 ceph-mon[129184]: pgmap v258: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 290 op/s 2023-07-14T19:36:53.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:53 smithi133 ceph-mon[128894]: pgmap v259: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-07-14T19:36:53.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:53 smithi082 ceph-mon[127461]: pgmap v259: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-07-14T19:36:53.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:53 smithi161 ceph-mon[129184]: pgmap v259: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2023-07-14T19:36:55.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:55 smithi133 ceph-mon[128894]: pgmap v260: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2023-07-14T19:36:55.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:55 smithi082 ceph-mon[127461]: pgmap v260: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2023-07-14T19:36:55.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:55 smithi161 ceph-mon[129184]: pgmap v260: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2023-07-14T19:36:57.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:57 smithi133 ceph-mon[128894]: pgmap v261: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 365 op/s 2023-07-14T19:36:57.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:57 smithi082 ceph-mon[127461]: pgmap v261: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 365 op/s 2023-07-14T19:36:57.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:57 smithi161 ceph-mon[129184]: pgmap v261: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 365 op/s 2023-07-14T19:36:59.446 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:36:59 smithi133 ceph-mon[128894]: pgmap v262: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 375 op/s 2023-07-14T19:36:59.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:36:59 smithi082 ceph-mon[127461]: pgmap v262: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 375 op/s 2023-07-14T19:36:59.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:36:59 smithi161 ceph-mon[129184]: pgmap v262: 97 pgs: 97 active+clean; 1.8 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 375 op/s 2023-07-14T19:37:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/540685297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 msgr2=0x7fbfa4104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/540685297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa4104900 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbf980099c0 tx=0x7fbf98009cd0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/540685297 shutdown_connections 2023-07-14T19:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/540685297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa4104fe0 0x7fbfa410b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/540685297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfa40ffa10 0x7fbfa40ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/540685297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa4104900 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/540685297 >> 172.21.15.82:0/540685297 conn(0x7fbfa4074cc0 msgr2=0x7fbfa40750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/540685297 shutdown_connections 2023-07-14T19:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.963+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/540685297 wait complete. 2023-07-14T19:37:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 Processor -- start 2023-07-14T19:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 -- start start 2023-07-14T19:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa40ffa10 0x7fbfa4195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa41962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfa4104fe0 0x7fbfa419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419af60 con 0x7fbfa41003b0 2023-07-14T19:37:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419b0d0 con 0x7fbfa4104fe0 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa9e5a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa419b290 con 0x7fbfa40ffa10 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa41962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa41962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44640/0 (socket says 172.21.15.82:44640) 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.964+0000 7fbfa2ffd700 1 -- 172.21.15.82:0/526894418 learned_addr learned my addr 172.21.15.82:0/526894418 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfa4104fe0 0x7fbfa419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa40ffa10 0x7fbfa4195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa2ffd700 1 -- 172.21.15.82:0/526894418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa40ffa10 msgr2=0x7fbfa4195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa40ffa10 0x7fbfa4195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa2ffd700 1 -- 172.21.15.82:0/526894418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfa4104fe0 msgr2=0x7fbfa419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfa4104fe0 0x7fbfa419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa2ffd700 1 -- 172.21.15.82:0/526894418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980095f0 con 0x7fbfa41003b0 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa41962b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa0ff9700 1 -- 172.21.15.82:0/526894418 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9801d070 con 0x7fbfa41003b0 2023-07-14T19:37:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa419b510 con 0x7fbfa41003b0 2023-07-14T19:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa0ff9700 1 -- 172.21.15.82:0/526894418 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf98005020 con 0x7fbfa41003b0 2023-07-14T19:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa0ff9700 1 -- 172.21.15.82:0/526894418 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98004a90 con 0x7fbfa41003b0 2023-07-14T19:37:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.965+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa41a2db0 con 0x7fbfa41003b0 2023-07-14T19:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.967+0000 7fbfa0ff9700 1 -- 172.21.15.82:0/526894418 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbf980054b0 con 0x7fbfa41003b0 2023-07-14T19:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.968+0000 7fbfa0ff9700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.968+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.968+0000 7fbfa0ff9700 1 -- 172.21.15.82:0/526894418 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fbf980cd700 con 0x7fbfa41003b0 2023-07-14T19:37:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.968+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009c60 tx=0x7fbf94009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.968+0000 7fbfb5ff2b80 1 Processor -- start 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.968+0000 7fbfb5ff2b80 1 -- start start 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1f00 0x560e1c1cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c0d0c50 con 0x560e1c0276b0 2023-07-14T19:37:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1239e0 con 0x560e1c1d1f00 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c11ce70 con 0x560e1c1d1a90 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44646/0 (socket says 172.21.15.82:44646) 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 -- 172.21.15.82:0/1232178445 learned_addr learned my addr 172.21.15.82:0/1232178445 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1f00 0x560e1c1cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 -- 172.21.15.82:0/1232178445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 msgr2=0x560e1c1cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 -- 172.21.15.82:0/1232178445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1f00 msgr2=0x560e1c1cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1f00 0x560e1c1cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 -- 172.21.15.82:0/1232178445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e1bf4ad80 con 0x560e1c0276b0 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d1550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e4e0 tx=0x7fbf94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2757413885996f7c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbf89ffb700 1 -- 172.21.15.82:0/1232178445 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400ee00 con 0x560e1c0276b0 2023-07-14T19:37:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbf89ffb700 1 -- 172.21.15.82:0/1232178445 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf94021070 con 0x560e1c0276b0 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.969+0000 7fbf89ffb700 1 -- 172.21.15.82:0/1232178445 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9401d7f0 con 0x560e1c0276b0 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/1232178445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 msgr2=0x560e1c1d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d1550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e4e0 tx=0x7fbf94014000 comp rx=0 tx=0).stop 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/1232178445 shutdown_connections 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1f00 0x560e1c1cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d1550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/1232178445 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/1232178445 >> 172.21.15.82:0/1232178445 conn(0x560e1c1c13d0 msgr2=0x560e1c1c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/1232178445 shutdown_connections 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/1232178445 wait complete. 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.970+0000 7fbfb5ff2b80 1 -- start start 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 Processor -- start 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1a90 0x560e1c1d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1f00 0x560e1c1d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c0bca60 con 0x560e1c0276b0 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c0bc760 con 0x560e1c1d1a90 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1bfa08c0 con 0x560e1c1d1f00 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfb5ff2b80 10 client.0 _ll_register_callbacks cb 0x560e1c0ba700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44652/0 (socket says 172.21.15.82:44652) 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfa37fe700 1 -- 172.21.15.82:0/387394693 learned_addr learned my addr 172.21.15.82:0/387394693 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1a90 0x560e1c1d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.971+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1f00 0x560e1c1d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa37fe700 1 -- 172.21.15.82:0/387394693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1f00 msgr2=0x560e1c1d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1f00 0x560e1c1d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa37fe700 1 -- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1a90 msgr2=0x560e1c1d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:00.995 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1a90 0x560e1c1d4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa37fe700 1 -- 172.21.15.82:0/387394693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf94017070 con 0x560e1c0276b0 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1a90 0x560e1c1d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1f00 0x560e1c1d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d7d70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ac1d7ee2dbfdbc server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbf82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbf82ffd700 1 -- 172.21.15.82:0/387394693 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94005350 con 0x560e1c0276b0 2023-07-14T19:37:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbf82ffd700 1 -- 172.21.15.82:0/387394693 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fbf940054b0 con 0x560e1c0276b0 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e1c076490 con 0x560e1c0276b0 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbf82ffd700 1 -- 172.21.15.82:0/387394693 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf94005670 con 0x560e1c0276b0 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfb5ff2b80 10 client.16206 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560e1bf33c00 con 0x560e1c0276b0 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.972+0000 7fbf82ffd700 1 -- 172.21.15.82:0/387394693 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e1c076490 con 0x560e1c0276b0 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbf82ffd700 1 -- 172.21.15.82:0/387394693 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbf9401c710 con 0x560e1c0276b0 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfb5ff2b80 10 client.16206 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfb5ff2b80 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfb5ff2b80 20 client.16206 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c1e51b0 0x560e1c1e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfb5ff2b80 4 client.16206 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e1c19ad40 con 0x560e1c1e51b0 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c1e51b0 0x560e1c1e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbf82ffd700 10 client.16206 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.973+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c1e51b0 0x560e1c1e75c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=456ff16ddae81ff9 server_cookie=67fb9106c9eff248 in_seq=0 out_seq=0 2023-07-14T19:37:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.976+0000 7fbf82ffd700 1 -- 172.21.15.82:0/387394693 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x560e1c19ad40 con 0x560e1c1e51b0 2023-07-14T19:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.976+0000 7fbf82ffd700 10 client.16206 handle_command_reply: tid=0 2023-07-14T19:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.976+0000 7fbfb5ff2b80 1 client.16206 shutdown 2023-07-14T19:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 msgr2=0x560e1c1d7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d7d70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400edf0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 shutdown_connections 2023-07-14T19:37:01.000 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c1e51b0 0x560e1c1e75c0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1f00 0x560e1c1d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c0276b0 0x560e1c1d7d70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbf82ffd700 0 client.16206 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/387394693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c1d1a90 0x560e1c1d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 >> 172.21.15.82:0/387394693 conn(0x560e1c1c13d0 msgr2=0x560e1c1c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 shutdown_connections 2023-07-14T19:37:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/387394693 wait complete. 2023-07-14T19:37:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.977+0000 7fbfb5ff2b80 20 client.16206 trim_cache size 0 max 16384 2023-07-14T19:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 Processor -- start 2023-07-14T19:37:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 -- start start 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1ec3c0 con 0x560e1c1d1f00 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1d5b90 con 0x560e1c0276b0 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1c7320 con 0x560e1c1d1a90 2023-07-14T19:37:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44664/0 (socket says 172.21.15.82:44664) 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 -- 172.21.15.82:0/3258389404 learned_addr learned my addr 172.21.15.82:0/3258389404 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 -- 172.21.15.82:0/3258389404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 msgr2=0x560e1c1dc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1dc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.008 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 -- 172.21.15.82:0/3258389404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 msgr2=0x560e1c1d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1d4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 -- 172.21.15.82:0/3258389404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e1bf711f0 con 0x560e1c1d1f00 2023-07-14T19:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1dc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.992+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1c63b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009d40 tx=0x7fbf9c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323d238bcedda14d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbf8a7fc700 1 -- 172.21.15.82:0/3258389404 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c007d80 con 0x560e1c1d1f00 2023-07-14T19:37:01.009 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbf8a7fc700 1 -- 172.21.15.82:0/3258389404 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c00f040 con 0x560e1c1d1f00 2023-07-14T19:37:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbf8a7fc700 1 -- 172.21.15.82:0/3258389404 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0207f0 con 0x560e1c1d1f00 2023-07-14T19:37:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/3258389404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 msgr2=0x560e1c1c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1c63b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009d40 tx=0x7fbf9c009d70 comp rx=0 tx=0).stop 2023-07-14T19:37:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/3258389404 shutdown_connections 2023-07-14T19:37:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1c63b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1d4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/3258389404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1dc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/3258389404 >> 172.21.15.82:0/3258389404 conn(0x560e1c1c13d0 msgr2=0x560e1c1d48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/3258389404 shutdown_connections 2023-07-14T19:37:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.993+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/3258389404 wait complete. 2023-07-14T19:37:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 -- start start 2023-07-14T19:37:01.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 Processor -- start 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1cbbb0 con 0x560e1c1d1f00 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1cbd20 con 0x560e1c0276b0 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e1c1cbee0 con 0x560e1c1d1a90 2023-07-14T19:37:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfb5ff2b80 10 client.0 _ll_register_callbacks cb 0x7fbf9c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41014/0 (socket says 172.21.15.82:41014) 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44670/0 (socket says 172.21.15.82:44670) 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfa37fe700 1 -- 172.21.15.82:0/2388479551 learned_addr learned my addr 172.21.15.82:0/2388479551 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.994+0000 7fbfa2ffd700 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfa3fff700 1 -- 172.21.15.82:0/2388479551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 msgr2=0x560e1c1caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfa3fff700 1 -- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 msgr2=0x560e1c1ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfa3fff700 1 -- 172.21.15.82:0/2388479551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0099f0 con 0x560e1c1d1f00 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfa3fff700 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1cb470 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c000f80 tx=0x7fbf9c004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2534140c689e7c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbf82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbf82ffd700 1 -- 172.21.15.82:0/2388479551 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c029670 con 0x560e1c1d1f00 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbf82ffd700 1 -- 172.21.15.82:0/2388479551 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fbf9c0297d0 con 0x560e1c1d1f00 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf94021070 con 0x560e1c1d1f00 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbf82ffd700 1 -- 172.21.15.82:0/2388479551 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9c004780 con 0x560e1c1d1f00 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 10 client.16212 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560e1bf49250 con 0x560e1c1d1f00 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbf82ffd700 1 -- 172.21.15.82:0/2388479551 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf94021070 con 0x560e1c1d1f00 2023-07-14T19:37:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbf82ffd700 1 -- 172.21.15.82:0/2388479551 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbf9c033c30 con 0x560e1c1d1f00 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 10 client.16212 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 20 client.16212 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c2399d0 0x560e1c23bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 4 client.16212 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.995+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e1c115e50 con 0x560e1c2399d0 2023-07-14T19:37:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.996+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c2399d0 0x560e1c23bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.996+0000 7fbf82ffd700 10 client.16212 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:00.996+0000 7fbfa37fe700 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c2399d0 0x560e1c23bde0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266b2630ef84c7df server_cookie=808c1dfbdbdaac99 in_seq=0 out_seq=0 2023-07-14T19:37:01.481 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:01 smithi082 ceph-mon[127461]: pgmap v263: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 395 op/s 2023-07-14T19:37:01.509 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:01 smithi161 ceph-mon[129184]: pgmap v263: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 395 op/s 2023-07-14T19:37:01.567 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:01 smithi133 ceph-mon[128894]: pgmap v263: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 395 op/s 2023-07-14T19:37:03.477 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:03 smithi082 ceph-mon[127461]: pgmap v264: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 407 op/s 2023-07-14T19:37:03.505 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:03 smithi161 ceph-mon[129184]: pgmap v264: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 407 op/s 2023-07-14T19:37:03.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:03 smithi133 ceph-mon[128894]: pgmap v264: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 407 op/s 2023-07-14T19:37:05.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:05 smithi082 ceph-mon[127461]: pgmap v265: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 346 op/s 2023-07-14T19:37:05.505 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:05 smithi161 ceph-mon[129184]: pgmap v265: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 346 op/s 2023-07-14T19:37:05.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:05 smithi133 ceph-mon[128894]: pgmap v265: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 346 op/s 2023-07-14T19:37:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.237+0000 7fbf82ffd700 1 -- 172.21.15.82:0/2388479551 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e1c115e50 con 0x560e1c2399d0 2023-07-14T19:37:06.251 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.237+0000 7fbf82ffd700 10 client.16212 handle_command_reply: tid=0 2023-07-14T19:37:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.237+0000 7fbfb5ff2b80 1 client.16212 shutdown 2023-07-14T19:37:06.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.239+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 msgr2=0x560e1c1cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:06.253 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.239+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1cb470 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c000f80 tx=0x7fbf9c004180 comp rx=0 tx=0).stop 2023-07-14T19:37:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.245+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 shutdown_connections 2023-07-14T19:37:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.245+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560e1c1d1f00 0x560e1c1cb470 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.245+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560e1c0276b0 0x560e1c1ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.245+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x560e1c2399d0 0x560e1c23bde0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.245+0000 7fbfb5ff2b80 1 --2- 172.21.15.82:0/2388479551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560e1c1d1a90 0x560e1c1caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.245+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 >> 172.21.15.82:0/2388479551 conn(0x560e1c1c13d0 msgr2=0x560e1c1d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.247+0000 7fbf82ffd700 0 client.16212 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:06.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.250+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 shutdown_connections 2023-07-14T19:37:06.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.250+0000 7fbfb5ff2b80 1 -- 172.21.15.82:0/2388479551 wait complete. 2023-07-14T19:37:06.264 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:37:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:37:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:37:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:37:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.250+0000 7fbfb5ff2b80 20 client.16212 trim_cache size 0 max 16384 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fbf8c0a12c0 msgr2=0x7fbf8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009c60 tx=0x7fbf94009260 comp rx=0 tx=0).stop 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 msgr2=0x7fbfa41962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa41962b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98009580 comp rx=0 tx=0).stop 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 shutdown_connections 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fbf8c0a12c0 0x7fbf8c0a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbfa4104fe0 0x7fbfa419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbfa40ffa10 0x7fbfa4195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 --2- 172.21.15.82:0/526894418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa41003b0 0x7fbfa41962b0 secure :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009fd0 tx=0x7fbf98009580 comp rx=0 tx=0).stop 2023-07-14T19:37:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.252+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 >> 172.21.15.82:0/526894418 conn(0x7fbfa4074cc0 msgr2=0x7fbfa4067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.253+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 shutdown_connections 2023-07-14T19:37:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:06.253+0000 7fbfa9e5a700 1 -- 172.21.15.82:0/526894418 wait complete. 2023-07-14T19:37:07.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:07 smithi082 ceph-mon[127461]: pgmap v266: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 439 op/s 2023-07-14T19:37:07.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:07 smithi082 ceph-mon[127461]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T19:37:07.505 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:07 smithi161 ceph-mon[129184]: pgmap v266: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 439 op/s 2023-07-14T19:37:07.505 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:07 smithi161 ceph-mon[129184]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T19:37:07.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:07 smithi133 ceph-mon[128894]: pgmap v266: 97 pgs: 97 active+clean; 1.8 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 439 op/s 2023-07-14T19:37:07.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:07 smithi133 ceph-mon[128894]: osdmap e92: 12 total, 12 up, 12 in 2023-07-14T19:37:09.506 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:09 smithi161 ceph-mon[129184]: pgmap v268: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 336 op/s 2023-07-14T19:37:09.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:09 smithi133 ceph-mon[128894]: pgmap v268: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 336 op/s 2023-07-14T19:37:09.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:09 smithi082 ceph-mon[127461]: pgmap v268: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 336 op/s 2023-07-14T19:37:11.507 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:11 smithi161 ceph-mon[129184]: pgmap v269: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 288 op/s 2023-07-14T19:37:11.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:11 smithi133 ceph-mon[128894]: pgmap v269: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 288 op/s 2023-07-14T19:37:11.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:11 smithi082 ceph-mon[127461]: pgmap v269: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 288 op/s 2023-07-14T19:37:13.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:13 smithi133 ceph-mon[128894]: pgmap v270: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-07-14T19:37:13.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:13 smithi082 ceph-mon[127461]: pgmap v270: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-07-14T19:37:13.762 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:13 smithi161 ceph-mon[129184]: pgmap v270: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2023-07-14T19:37:15.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:15 smithi133 ceph-mon[128894]: pgmap v271: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2023-07-14T19:37:15.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:15 smithi082 ceph-mon[127461]: pgmap v271: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2023-07-14T19:37:15.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:15 smithi161 ceph-mon[129184]: pgmap v271: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2023-07-14T19:37:16.452 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:16 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:37:16.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:16 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:37:16.755 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:16 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:37:17.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:17 smithi133 ceph-mon[128894]: pgmap v272: 97 pgs: 30 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 52 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 82 op/s 2023-07-14T19:37:17.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:17 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:37:17.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:17 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:37:17.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:17 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:37:17.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:17 smithi082 ceph-mon[127461]: pgmap v272: 97 pgs: 30 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 52 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 82 op/s 2023-07-14T19:37:17.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:17 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:37:17.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:17 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:37:17.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:17 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:37:17.755 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:17 smithi161 ceph-mon[129184]: pgmap v272: 97 pgs: 30 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 52 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 82 op/s 2023-07-14T19:37:17.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:17 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:37:17.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:17 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:37:17.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:17 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:37:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 -- 172.21.15.82:0/2187791934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67181003b0 msgr2=0x7f6718104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 --2- 172.21.15.82:0/2187791934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67181003b0 0x7f6718104900 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6708009990 tx=0x7f6708009ca0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 -- 172.21.15.82:0/2187791934 shutdown_connections 2023-07-14T19:37:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 --2- 172.21.15.82:0/2187791934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718104fe0 0x7f671810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 --2- 172.21.15.82:0/2187791934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67180ffa10 0x7f67180ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 --2- 172.21.15.82:0/2187791934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67181003b0 0x7f6718104900 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 -- 172.21.15.82:0/2187791934 >> 172.21.15.82:0/2187791934 conn(0x7f6718074cc0 msgr2=0x7f67180750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 -- 172.21.15.82:0/2187791934 shutdown_connections 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.418+0000 7f671da89700 1 -- 172.21.15.82:0/2187791934 wait complete. 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.419+0000 7f671da89700 1 Processor -- start 2023-07-14T19:37:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.419+0000 7f671da89700 1 -- start start 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f671da89700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67180ffa10 0x7f6718197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f671da89700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181003b0 0x7f67181984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f671da89700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718104fe0 0x7f671819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f671da89700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671819d190 con 0x7f67180ffa10 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f671da89700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671819d300 con 0x7f6718104fe0 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f671da89700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671819d4c0 con 0x7f67181003b0 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f67177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67180ffa10 0x7f6718197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f67177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67180ffa10 0x7f6718197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44680/0 (socket says 172.21.15.82:44680) 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f67177fe700 1 -- 172.21.15.82:0/1877129046 learned_addr learned my addr 172.21.15.82:0/1877129046 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f6716ffd700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181003b0 0x7f67181984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f6716ffd700 1 -- 172.21.15.82:0/1877129046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718104fe0 msgr2=0x7f671819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f6716ffd700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718104fe0 0x7f671819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f6716ffd700 1 -- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67180ffa10 msgr2=0x7f6718197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f6716ffd700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67180ffa10 0x7f6718197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.420+0000 7f6716ffd700 1 -- 172.21.15.82:0/1877129046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67080095f0 con 0x7f67181003b0 2023-07-14T19:37:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.421+0000 7f6716ffd700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181003b0 0x7f67181984e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67080098e0 tx=0x7f6708009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.421+0000 7f6714ff9700 1 -- 172.21.15.82:0/1877129046 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670801d070 con 0x7f67181003b0 2023-07-14T19:37:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.421+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671819d740 con 0x7f67181003b0 2023-07-14T19:37:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.421+0000 7f6714ff9700 1 -- 172.21.15.82:0/1877129046 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67080047c0 con 0x7f67181003b0 2023-07-14T19:37:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.421+0000 7f6714ff9700 1 -- 172.21.15.82:0/1877129046 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67080053a0 con 0x7f67181003b0 2023-07-14T19:37:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.421+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67181a4fe0 con 0x7f67181003b0 2023-07-14T19:37:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.425+0000 7f6714ff9700 1 -- 172.21.15.82:0/1877129046 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6708004930 con 0x7f67181003b0 2023-07-14T19:37:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.425+0000 7f6714ff9700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f67000a13c0 0x7f67000a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.426+0000 7f67177fe700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f67000a13c0 0x7f67000a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.426+0000 7f6729c21b80 1 Processor -- start 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.426+0000 7f6714ff9700 1 -- 172.21.15.82:0/1877129046 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f67080cdd10 con 0x7f67181003b0 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.426+0000 7f67177fe700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f67000a13c0 0x7f67000a3870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f670c009cb0 tx=0x7f670c009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 -- start start 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5ded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df2a0 0x5588be5d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 0x5588be5d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be4d5c50 con 0x5588be5df710 2023-07-14T19:37:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5289e0 con 0x5588be5df2a0 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6729c21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be521e70 con 0x5588be42c6b0 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6717fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 0x5588be5d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6717fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 0x5588be5d9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44690/0 (socket says 172.21.15.82:44690) 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6717fff700 1 -- 172.21.15.82:0/784157406 learned_addr learned my addr 172.21.15.82:0/784157406 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f67177fe700 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5ded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6717fff700 1 -- 172.21.15.82:0/784157406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 msgr2=0x5588be5ded60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.427+0000 7f6717fff700 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5ded60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f6717fff700 1 -- 172.21.15.82:0/784157406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df2a0 msgr2=0x5588be5d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f6717fff700 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df2a0 0x5588be5d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f6717fff700 1 -- 172.21.15.82:0/784157406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588be34fd80 con 0x5588be5df710 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f6717fff700 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 0x5588be5d9790 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f671000e380 tx=0x7f671000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191e2d5433ebb358 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f66fdffb700 1 -- 172.21.15.82:0/784157406 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710018070 con 0x5588be5df710 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f66fdffb700 1 -- 172.21.15.82:0/784157406 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671000ee30 con 0x5588be5df710 2023-07-14T19:37:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.428+0000 7f66fdffb700 1 -- 172.21.15.82:0/784157406 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710010040 con 0x5588be5df710 2023-07-14T19:37:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 -- 172.21.15.82:0/784157406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 msgr2=0x5588be5d9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 0x5588be5d9790 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f671000e380 tx=0x7f671000b040 comp rx=0 tx=0).stop 2023-07-14T19:37:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 -- 172.21.15.82:0/784157406 shutdown_connections 2023-07-14T19:37:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df710 0x5588be5d9790 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5ded60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 --2- 172.21.15.82:0/784157406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df2a0 0x5588be5d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.429+0000 7f6729c21b80 1 -- 172.21.15.82:0/784157406 >> 172.21.15.82:0/784157406 conn(0x5588be5c63d0 msgr2=0x5588be5cb530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.430+0000 7f6729c21b80 1 -- 172.21.15.82:0/784157406 shutdown_connections 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.430+0000 7f6729c21b80 1 -- 172.21.15.82:0/784157406 wait complete. 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.430+0000 7f6729c21b80 1 -- start start 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.430+0000 7f6729c21b80 1 Processor -- start 2023-07-14T19:37:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df2a0 0x5588be5e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5e2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be4c1a60 con 0x5588be5df2a0 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be4c1760 con 0x5588be5df710 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be3a58c0 con 0x5588be42c6b0 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6729c21b80 10 client.0 _ll_register_callbacks cb 0x5588be4bf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6717fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5e2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6717fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5e2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41042/0 (socket says 172.21.15.82:41042) 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f6717fff700 1 -- 172.21.15.82:0/4034304761 learned_addr learned my addr 172.21.15.82:0/4034304761 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f67177fe700 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.431+0000 7f67177fe700 1 -- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 msgr2=0x5588be5e2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.432+0000 7f67177fe700 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5e2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.432+0000 7f67177fe700 1 -- 172.21.15.82:0/4034304761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df2a0 msgr2=0x5588be5e8b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.432+0000 7f67177fe700 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df2a0 0x5588be5e8b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.432+0000 7f67177fe700 1 -- 172.21.15.82:0/4034304761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671000e0c0 con 0x5588be42c6b0 2023-07-14T19:37:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.432+0000 7f67177fe700 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5e85f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f670c009b70 tx=0x7f670c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f038c81e7eeddb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.434+0000 7f66f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:37:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.434+0000 7f66f6ffd700 1 -- 172.21.15.82:0/4034304761 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c01e810 con 0x5588be42c6b0 2023-07-14T19:37:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.434+0000 7f66f6ffd700 1 -- 172.21.15.82:0/4034304761 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f670c018070 con 0x5588be42c6b0 2023-07-14T19:37:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.434+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588be47b490 con 0x5588be42c6b0 2023-07-14T19:37:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f66f6ffd700 1 -- 172.21.15.82:0/4034304761 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f670c01e970 con 0x5588be42c6b0 2023-07-14T19:37:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 10 client.25774 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5588be338c00 con 0x5588be42c6b0 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f66f6ffd700 1 -- 172.21.15.82:0/4034304761 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588be47b490 con 0x5588be42c6b0 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f66f6ffd700 1 -- 172.21.15.82:0/4034304761 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f670c028430 con 0x5588be42c6b0 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 20 client.25774 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:19.456 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be5f2b10 0x5588be5f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 4 client.25774 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.435+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588be59fd40 con 0x5588be5f2b10 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.436+0000 7f6716ffd700 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be5f2b10 0x5588be5f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.436+0000 7f66f6ffd700 10 client.25774 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.436+0000 7f6716ffd700 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be5f2b10 0x5588be5f4f20 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30d4a3b6ed65a7e server_cookie=4d574d8d87ed2955 in_seq=0 out_seq=0 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.439+0000 7f66f6ffd700 1 -- 172.21.15.82:0/4034304761 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5588be59fd40 con 0x5588be5f2b10 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.439+0000 7f66f6ffd700 10 client.25774 handle_command_reply: tid=0 2023-07-14T19:37:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.439+0000 7f6729c21b80 1 client.25774 shutdown 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 msgr2=0x5588be5e85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5e85f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f670c009b70 tx=0x7f670c0047c0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 shutdown_connections 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5e2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be5f2b10 0x5588be5f4f20 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be42c6b0 0x5588be5e85f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f66f6ffd700 0 client.25774 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:19.458 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 --2- 172.21.15.82:0/4034304761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be5df2a0 0x5588be5e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 >> 172.21.15.82:0/4034304761 conn(0x5588be5c63d0 msgr2=0x5588be5cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 shutdown_connections 2023-07-14T19:37:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 1 -- 172.21.15.82:0/4034304761 wait complete. 2023-07-14T19:37:19.459 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.440+0000 7f6729c21b80 20 client.25774 trim_cache size 0 max 16384 2023-07-14T19:37:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.463+0000 7f6729c21b80 1 Processor -- start 2023-07-14T19:37:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 -- start start 2023-07-14T19:37:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be5e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5df2a0 0x5588be5cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5da1e0 con 0x5588be42c6b0 2023-07-14T19:37:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5e0da0 con 0x5588be5df710 2023-07-14T19:37:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6729c21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5cd480 con 0x5588be5df2a0 2023-07-14T19:37:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f67177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be5e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f67177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be5e01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44700/0 (socket says 172.21.15.82:44700) 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f6717fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.464+0000 7f67177fe700 1 -- 172.21.15.82:0/3947590085 learned_addr learned my addr 172.21.15.82:0/3947590085 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f67177fe700 1 -- 172.21.15.82:0/3947590085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5df2a0 msgr2=0x5588be5cb3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f67177fe700 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5df2a0 0x5588be5cb3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f67177fe700 1 -- 172.21.15.82:0/3947590085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 msgr2=0x5588be5d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f67177fe700 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f67177fe700 1 -- 172.21.15.82:0/3947590085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588be3761f0 con 0x5588be42c6b0 2023-07-14T19:37:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f67177fe700 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be5e01f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f670c0287c0 tx=0x7f670c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6599b64306b5f12 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f66fe7fc700 1 -- 172.21.15.82:0/3947590085 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c038070 con 0x5588be42c6b0 2023-07-14T19:37:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f66fe7fc700 1 -- 172.21.15.82:0/3947590085 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f670c023070 con 0x5588be42c6b0 2023-07-14T19:37:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.465+0000 7f66fe7fc700 1 -- 172.21.15.82:0/3947590085 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c01edf0 con 0x5588be42c6b0 2023-07-14T19:37:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 -- 172.21.15.82:0/3947590085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 msgr2=0x5588be5e01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be5e01f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f670c0287c0 tx=0x7f670c011a50 comp rx=0 tx=0).stop 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 -- 172.21.15.82:0/3947590085 shutdown_connections 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be5d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be5e01f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3947590085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5df2a0 0x5588be5cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.466+0000 7f6729c21b80 1 -- 172.21.15.82:0/3947590085 >> 172.21.15.82:0/3947590085 conn(0x5588be5c63d0 msgr2=0x5588be5dfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.468+0000 7f6729c21b80 1 -- 172.21.15.82:0/3947590085 shutdown_connections 2023-07-14T19:37:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.468+0000 7f6729c21b80 1 -- 172.21.15.82:0/3947590085 wait complete. 2023-07-14T19:37:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.468+0000 7f6729c21b80 1 -- start start 2023-07-14T19:37:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 Processor -- start 2023-07-14T19:37:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be641ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be642020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5e80b0 0x5588be5e8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5e8c30 con 0x5588be42c6b0 2023-07-14T19:37:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5e8da0 con 0x5588be5df710 2023-07-14T19:37:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588be5e6780 con 0x5588be5e80b0 2023-07-14T19:37:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6729c21b80 10 client.0 _ll_register_callbacks cb 0x5588be4bf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f67177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be641ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f67177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be641ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44704/0 (socket says 172.21.15.82:44704) 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f67177fe700 1 -- 172.21.15.82:0/3728598063 learned_addr learned my addr 172.21.15.82:0/3728598063 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6716ffd700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be642020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.469+0000 7f6717fff700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5e80b0 0x5588be5e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f67177fe700 1 -- 172.21.15.82:0/3728598063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5e80b0 msgr2=0x5588be5e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f67177fe700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5e80b0 0x5588be5e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f67177fe700 1 -- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 msgr2=0x5588be642020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f67177fe700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be642020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f67177fe700 1 -- 172.21.15.82:0/3728598063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588be338c00 con 0x5588be42c6b0 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f67177fe700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be641ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f670c009b70 tx=0x7f670c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9932c0b2c149b56e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f66f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f66f6ffd700 1 -- 172.21.15.82:0/3728598063 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c038040 con 0x5588be42c6b0 2023-07-14T19:37:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f66f6ffd700 1 -- 172.21.15.82:0/3728598063 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f670c018050 con 0x5588be42c6b0 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f671000ee30 con 0x5588be42c6b0 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f66f6ffd700 1 -- 172.21.15.82:0/3728598063 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f670c011e00 con 0x5588be42c6b0 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f6729c21b80 10 client.16227 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.470+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5588be34e250 con 0x5588be42c6b0 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f66f6ffd700 1 -- 172.21.15.82:0/3728598063 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671000ee30 con 0x5588be42c6b0 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f66f6ffd700 1 -- 172.21.15.82:0/3728598063 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f670c01e700 con 0x5588be42c6b0 2023-07-14T19:37:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6729c21b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6729c21b80 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6729c21b80 20 client.16227 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be63f9e0 0x5588be63fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6729c21b80 4 client.16227 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588be51ae50 con 0x5588be63f9e0 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.471+0000 7f6716ffd700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be63f9e0 0x5588be63fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.472+0000 7f66f6ffd700 10 client.16227 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.472+0000 7f6716ffd700 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be63f9e0 0x5588be63fde0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c94ac36b736b27 server_cookie=f3221e093d3f56c7 in_seq=0 out_seq=0 2023-07-14T19:37:19.697 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:19 smithi133 ceph-mon[128894]: pgmap v273: 97 pgs: 22 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 98 op/s 2023-07-14T19:37:19.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:19 smithi082 ceph-mon[127461]: pgmap v273: 97 pgs: 22 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 98 op/s 2023-07-14T19:37:19.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:19 smithi161 ceph-mon[129184]: pgmap v273: 97 pgs: 22 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 98 op/s 2023-07-14T19:37:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.877+0000 7f66f6ffd700 1 -- 172.21.15.82:0/3728598063 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588be51ae50 con 0x5588be63f9e0 2023-07-14T19:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.877+0000 7f66f6ffd700 10 client.16227 handle_command_reply: tid=0 2023-07-14T19:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.877+0000 7f6729c21b80 1 client.16227 shutdown 2023-07-14T19:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.877+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 msgr2=0x5588be641ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.877+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be641ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f670c009b70 tx=0x7f670c009eb0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 shutdown_connections 2023-07-14T19:37:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5588be63f9e0 0x5588be63fde0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5588be5e80b0 0x5588be5e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588be42c6b0 0x5588be641ae0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 --2- 172.21.15.82:0/3728598063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5588be5df710 0x5588be642020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 >> 172.21.15.82:0/3728598063 conn(0x5588be5c63d0 msgr2=0x5588be5e0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f66f6ffd700 0 client.16227 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 shutdown_connections 2023-07-14T19:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 1 -- 172.21.15.82:0/3728598063 wait complete. 2023-07-14T19:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.878+0000 7f6729c21b80 20 client.16227 trim_cache size 0 max 16384 2023-07-14T19:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:37:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:37:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f67000a13c0 msgr2=0x7f67000a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f67000a13c0 0x7f67000a3870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f670c009cb0 tx=0x7f670c009260 comp rx=0 tx=0).stop 2023-07-14T19:37:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181003b0 msgr2=0x7f67181984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181003b0 0x7f67181984e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f67080098e0 tx=0x7f6708009580 comp rx=0 tx=0).stop 2023-07-14T19:37:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 shutdown_connections 2023-07-14T19:37:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f67000a13c0 0x7f67000a3870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6718104fe0 0x7f671819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67180ffa10 0x7f6718197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 --2- 172.21.15.82:0/1877129046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67181003b0 0x7f67181984e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 >> 172.21.15.82:0/1877129046 conn(0x7f6718074cc0 msgr2=0x7f6718107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 shutdown_connections 2023-07-14T19:37:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:19.879+0000 7f671da89700 1 -- 172.21.15.82:0/1877129046 wait complete. 2023-07-14T19:37:21.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:21 smithi133 ceph-mon[128894]: pgmap v274: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2023-07-14T19:37:21.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:21 smithi082 ceph-mon[127461]: pgmap v274: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2023-07-14T19:37:21.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:21 smithi161 ceph-mon[129184]: pgmap v274: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 134 op/s 2023-07-14T19:37:23.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:23 smithi133 ceph-mon[128894]: pgmap v275: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-07-14T19:37:23.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:23 smithi133 ceph-mon[128894]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T19:37:23.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:23 smithi082 ceph-mon[127461]: pgmap v275: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-07-14T19:37:23.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:23 smithi082 ceph-mon[127461]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T19:37:23.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:23 smithi161 ceph-mon[129184]: pgmap v275: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 217 op/s 2023-07-14T19:37:23.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:23 smithi161 ceph-mon[129184]: osdmap e93: 12 total, 12 up, 12 in 2023-07-14T19:37:25.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:25 smithi133 ceph-mon[128894]: pgmap v277: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2023-07-14T19:37:25.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:25 smithi082 ceph-mon[127461]: pgmap v277: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2023-07-14T19:37:25.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:25 smithi161 ceph-mon[129184]: pgmap v277: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2023-07-14T19:37:27.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:27 smithi133 ceph-mon[128894]: pgmap v278: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 343 op/s 2023-07-14T19:37:27.728 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:27 smithi082 ceph-mon[127461]: pgmap v278: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 343 op/s 2023-07-14T19:37:27.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:27 smithi161 ceph-mon[129184]: pgmap v278: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 343 op/s 2023-07-14T19:37:29.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:29 smithi133 ceph-mon[128894]: pgmap v279: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 373 op/s 2023-07-14T19:37:29.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:29 smithi082 ceph-mon[127461]: pgmap v279: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 373 op/s 2023-07-14T19:37:29.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:29 smithi161 ceph-mon[129184]: pgmap v279: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 373 op/s 2023-07-14T19:37:31.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:31 smithi133 ceph-mon[128894]: pgmap v280: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 391 op/s 2023-07-14T19:37:31.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:31 smithi082 ceph-mon[127461]: pgmap v280: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 391 op/s 2023-07-14T19:37:31.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:31 smithi161 ceph-mon[129184]: pgmap v280: 97 pgs: 97 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 391 op/s 2023-07-14T19:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.993+0000 7f1b2dd9b700 1 -- 172.21.15.82:0/2067950838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400bb10 con 0x7f1b30101810 2023-07-14T19:37:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 -- 172.21.15.82:0/2067950838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 msgr2=0x7f1b30109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 --2- 172.21.15.82:0/2067950838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b30109d00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009990 tx=0x7f1b24009ca0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 -- 172.21.15.82:0/2067950838 shutdown_connections 2023-07-14T19:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 --2- 172.21.15.82:0/2067950838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b3010a240 0x7f1b3010c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 --2- 172.21.15.82:0/2067950838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b30100f00 0x7f1b301012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 --2- 172.21.15.82:0/2067950838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b30109d00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 -- 172.21.15.82:0/2067950838 >> 172.21.15.82:0/2067950838 conn(0x7f1b30075d00 msgr2=0x7f1b30076100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 -- 172.21.15.82:0/2067950838 shutdown_connections 2023-07-14T19:37:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.994+0000 7f1b3520c700 1 -- 172.21.15.82:0/2067950838 wait complete. 2023-07-14T19:37:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.995+0000 7f1b3520c700 1 Processor -- start 2023-07-14T19:37:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 -- start start 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b30100f00 0x7f1b301aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b301ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b3010a240 0x7f1b301a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b301a5140 con 0x7f1b30101810 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b301a52b0 con 0x7f1b3010a240 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b3520c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b301a5470 con 0x7f1b30100f00 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b301ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b301ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44720/0 (socket says 172.21.15.82:44720) 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1496467613 learned_addr learned my addr 172.21.15.82:0/1496467613 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b30100f00 0x7f1b301aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.996+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b3010a240 0x7f1b301a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1496467613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b30100f00 msgr2=0x7f1b301aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b30100f00 0x7f1b301aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1496467613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b3010a240 msgr2=0x7f1b301a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b3010a240 0x7f1b301a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1496467613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240095f0 con 0x7f1b30101810 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b301ab070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1b24000c00 tx=0x7f1b2400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b17fff700 1 -- 172.21.15.82:0/1496467613 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2401d070 con 0x7f1b30101810 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b301a56f0 con 0x7f1b30101810 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b17fff700 1 -- 172.21.15.82:0/1496467613 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b2400be40 con 0x7f1b30101810 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b17fff700 1 -- 172.21.15.82:0/1496467613 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240173f0 con 0x7f1b30101810 2023-07-14T19:37:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:32.997+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b301b1ab0 con 0x7f1b30101810 2023-07-14T19:37:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 Processor -- start 2023-07-14T19:37:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 -- start start 2023-07-14T19:37:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 0x563d18e1d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1da90 0x563d18e179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e17f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b17fff700 1 -- 172.21.15.82:0/1496467613 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1b24022470 con 0x7f1b30101810 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18d1cc50 con 0x563d18c736b0 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18d6f9e0 con 0x563d18e1da90 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.001+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18d68e70 con 0x563d18e1df00 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 0x563d18e1d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 0x563d18e1d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44724/0 (socket says 172.21.15.82:44724) 2023-07-14T19:37:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 -- 172.21.15.82:0/2908646827 learned_addr learned my addr 172.21.15.82:0/2908646827 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b17fff700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b180a1330 0x7f1b180a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b17fff700 1 -- 172.21.15.82:0/1496467613 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f1b240cdf60 con 0x7f1b30101810 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b180a1330 0x7f1b180a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 -- 172.21.15.82:0/2908646827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 msgr2=0x563d18e17f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e17f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 -- 172.21.15.82:0/2908646827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1da90 msgr2=0x563d18e179e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1da90 0x563d18e179e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.002+0000 7f1b2f59e700 1 -- 172.21.15.82:0/2908646827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d18b96d80 con 0x563d18c736b0 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.003+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 0x563d18e1d550 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1b280079e0 tx=0x7f1b28007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c8136697c4c0cd server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.003+0000 7f1b157fa700 1 -- 172.21.15.82:0/2908646827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28018a80 con 0x563d18c736b0 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.003+0000 7f1b157fa700 1 -- 172.21.15.82:0/2908646827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b2800f050 con 0x563d18c736b0 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.003+0000 7f1b157fa700 1 -- 172.21.15.82:0/2908646827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280259b0 con 0x563d18c736b0 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b180a1330 0x7f1b180a37e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b28022000 tx=0x7f1b2801e000 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 -- 172.21.15.82:0/2908646827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 msgr2=0x563d18e1d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 0x563d18e1d550 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1b280079e0 tx=0x7f1b28007cf0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 -- 172.21.15.82:0/2908646827 shutdown_connections 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e17f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18c736b0 0x563d18e1d550 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/2908646827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1da90 0x563d18e179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 -- 172.21.15.82:0/2908646827 >> 172.21.15.82:0/2908646827 conn(0x563d18e0d3d0 msgr2=0x563d18e12530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 -- 172.21.15.82:0/2908646827 shutdown_connections 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 -- 172.21.15.82:0/2908646827 wait complete. 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.004+0000 7f1b413a4b80 1 -- start start 2023-07-14T19:37:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 Processor -- start 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e23d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1da90 0x563d18e21010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 0x563d18e215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18d08a60 con 0x563d18e1df00 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18d08760 con 0x563d18c736b0 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18bec8c0 con 0x563d18e1da90 2023-07-14T19:37:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b413a4b80 10 client.0 _ll_register_callbacks cb 0x563d18d06700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b2f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 0x563d18e215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b2f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 0x563d18e215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44730/0 (socket says 172.21.15.82:44730) 2023-07-14T19:37:33.026 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.005+0000 7f1b2f59e700 1 -- 172.21.15.82:0/1813495801 learned_addr learned my addr 172.21.15.82:0/1813495801 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b2f59e700 1 -- 172.21.15.82:0/1813495801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1da90 msgr2=0x563d18e21010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1da90 0x563d18e21010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b2f59e700 1 -- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 msgr2=0x563d18e23d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e23d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b2f59e700 1 -- 172.21.15.82:0/1813495801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b28007640 con 0x563d18e1df00 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 0x563d18e215d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1b28025880 tx=0x7f1b28006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cdd953cfb786b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1813495801 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28025970 con 0x563d18e1df00 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1813495801 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f1b28025e80 con 0x563d18e1df00 2023-07-14T19:37:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d18cc2490 con 0x563d18e1df00 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1813495801 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b28009400 con 0x563d18e1df00 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b413a4b80 10 client.16239 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.006+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x563d18b7fc00 con 0x563d18e1df00 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1813495801 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d18cc2490 con 0x563d18e1df00 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1813495801 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1b28038070 con 0x563d18e1df00 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b413a4b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b413a4b80 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b413a4b80 20 client.16239 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e31300 0x563d18e33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b413a4b80 4 client.16239 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.007+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d18de6d40 con 0x563d18e31300 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.009+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e31300 0x563d18e33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.010+0000 7f1b12ff5700 10 client.16239 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.010+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e31300 0x563d18e33790 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1714c5c64d361835 server_cookie=588d0859c40bd972 in_seq=0 out_seq=0 2023-07-14T19:37:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.013+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1813495801 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x563d18de6d40 con 0x563d18e31300 2023-07-14T19:37:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.013+0000 7f1b12ff5700 10 client.16239 handle_command_reply: tid=0 2023-07-14T19:37:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.013+0000 7f1b413a4b80 1 client.16239 shutdown 2023-07-14T19:37:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 msgr2=0x563d18e215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 0x563d18e215d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1b28025880 tx=0x7f1b28006000 comp rx=0 tx=0).stop 2023-07-14T19:37:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 shutdown_connections 2023-07-14T19:37:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1df00 0x563d18e215d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e23d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e31300 0x563d18e33790 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1813495801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1da90 0x563d18e21010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b12ff5700 0 client.16239 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 >> 172.21.15.82:0/1813495801 conn(0x563d18e0d3d0 msgr2=0x563d18e11c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 shutdown_connections 2023-07-14T19:37:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1813495801 wait complete. 2023-07-14T19:37:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.016+0000 7f1b413a4b80 20 client.16239 trim_cache size 0 max 16384 2023-07-14T19:37:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.038+0000 7f1b413a4b80 1 Processor -- start 2023-07-14T19:37:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 -- start start 2023-07-14T19:37:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 0x563d18e179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1df00 0x563d18e1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18e21fd0 con 0x563d18e1da90 2023-07-14T19:37:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18e2e930 con 0x563d18e1df00 2023-07-14T19:37:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18e0f450 con 0x563d18c736b0 2023-07-14T19:37:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 0x563d18e179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 0x563d18e179e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:53998/0 (socket says 172.21.15.82:53998) 2023-07-14T19:37:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 -- 172.21.15.82:0/3607482687 learned_addr learned my addr 172.21.15.82:0/3607482687 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1df00 0x563d18e1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 -- 172.21.15.82:0/3607482687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1df00 msgr2=0x563d18e1ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1df00 0x563d18e1ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 -- 172.21.15.82:0/3607482687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 msgr2=0x563d18e283b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e283b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2ed9d700 1 -- 172.21.15.82:0/3607482687 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d18bbd1f0 con 0x563d18c736b0 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.039+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e283b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.040+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 0x563d18e179e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x563d18e340b0 tx=0x7f1b20018c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e657c1d97d94972 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.040+0000 7f1b15ffb700 1 -- 172.21.15.82:0/3607482687 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20009db0 con 0x563d18c736b0 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.040+0000 7f1b15ffb700 1 -- 172.21.15.82:0/3607482687 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b20018d30 con 0x563d18c736b0 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.040+0000 7f1b15ffb700 1 -- 172.21.15.82:0/3607482687 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20021d70 con 0x563d18c736b0 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 -- 172.21.15.82:0/3607482687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 msgr2=0x563d18e179e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 0x563d18e179e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x563d18e340b0 tx=0x7f1b20018c10 comp rx=0 tx=0).stop 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 -- 172.21.15.82:0/3607482687 shutdown_connections 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18e1df00 0x563d18e1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18c736b0 0x563d18e179e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/3607482687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e283b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 -- 172.21.15.82:0/3607482687 >> 172.21.15.82:0/3607482687 conn(0x563d18e0d3d0 msgr2=0x563d18e1e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 -- 172.21.15.82:0/3607482687 shutdown_connections 2023-07-14T19:37:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.041+0000 7f1b413a4b80 1 -- 172.21.15.82:0/3607482687 wait complete. 2023-07-14T19:37:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 -- start start 2023-07-14T19:37:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 Processor -- start 2023-07-14T19:37:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e24ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e7e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18e7e5d0 con 0x563d18e1da90 2023-07-14T19:37:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18e7e740 con 0x563d18c736b0 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d18e7e900 con 0x563d18e1df00 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.042+0000 7f1b413a4b80 10 client.0 _ll_register_callbacks cb 0x7f1b28007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e24ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2f59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e7e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41088/0 (socket says 172.21.15.82:41088) 2023-07-14T19:37:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2ed9d700 1 -- 172.21.15.82:0/1528966827 learned_addr learned my addr 172.21.15.82:0/1528966827 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e7e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54008/0 (socket says 172.21.15.82:54008) 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e24ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44742/0 (socket says 172.21.15.82:44742) 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1528966827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 msgr2=0x563d18e7e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e7e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 msgr2=0x563d18e24580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e24580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2e59c700 1 -- 172.21.15.82:0/1528966827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b301b1ab0 con 0x563d18e1da90 2023-07-14T19:37:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.043+0000 7f1b2f59e700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e7e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:37:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.046+0000 7f1b2e59c700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e24ac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1b240cb6b0 tx=0x7f1b240cbc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa6eb2b3e98b112 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.046+0000 7f1b12ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.047+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1528966827 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2400fe20 con 0x563d18e1da90 2023-07-14T19:37:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.047+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1528966827 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f1b2409b0c0 con 0x563d18e1da90 2023-07-14T19:37:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b2800f050 con 0x563d18e1da90 2023-07-14T19:37:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1528966827 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b2409f0f0 con 0x563d18e1da90 2023-07-14T19:37:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 10 client.16245 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f1b200189d0 con 0x563d18e1da90 2023-07-14T19:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1528966827 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b2800f050 con 0x563d18e1da90 2023-07-14T19:37:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1528966827 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1b24022720 con 0x563d18e1da90 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 20 client.16245 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e96d80 0x563d18e99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 4 client.16245 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.050+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d18d61e50 con 0x563d18e96d80 2023-07-14T19:37:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.051+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e96d80 0x563d18e99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.052+0000 7f1b12ff5700 10 client.16245 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.052+0000 7f1b2ed9d700 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e96d80 0x563d18e99190 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed57beea8238e8e server_cookie=4dd6bd746306b79e in_seq=0 out_seq=0 2023-07-14T19:37:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.200+0000 7f1b12ff5700 1 -- 172.21.15.82:0/1528966827 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d18d61e50 con 0x563d18e96d80 2023-07-14T19:37:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.200+0000 7f1b12ff5700 10 client.16245 handle_command_reply: tid=0 2023-07-14T19:37:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 client.16245 shutdown 2023-07-14T19:37:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 msgr2=0x563d18e24ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e24ac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1b240cb6b0 tx=0x7f1b240cbc10 comp rx=0 tx=0).stop 2023-07-14T19:37:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 shutdown_connections 2023-07-14T19:37:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x563d18e96d80 0x563d18e99190 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563d18e1df00 0x563d18e7e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d18c736b0 0x563d18e24580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 --2- 172.21.15.82:0/1528966827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d18e1da90 0x563d18e24ac0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b12ff5700 0 client.16245 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 >> 172.21.15.82:0/1528966827 conn(0x563d18e0d3d0 msgr2=0x563d18e1eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 shutdown_connections 2023-07-14T19:37:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.201+0000 7f1b413a4b80 1 -- 172.21.15.82:0/1528966827 wait complete. 2023-07-14T19:37:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b413a4b80 20 client.16245 trim_cache size 0 max 16384 2023-07-14T19:37:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:37:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:37:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:37:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:37:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b180a1330 msgr2=0x7f1b180a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b180a1330 0x7f1b180a37e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b28022000 tx=0x7f1b2801e000 comp rx=0 tx=0).stop 2023-07-14T19:37:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 msgr2=0x7f1b301ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b301ab070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1b24000c00 tx=0x7f1b2400f690 comp rx=0 tx=0).stop 2023-07-14T19:37:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 shutdown_connections 2023-07-14T19:37:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1b180a1330 0x7f1b180a37e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b3010a240 0x7f1b301a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b30100f00 0x7f1b301aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 --2- 172.21.15.82:0/1496467613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b30101810 0x7f1b301ab070 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 >> 172.21.15.82:0/1496467613 conn(0x7f1b30075d00 msgr2=0x7f1b300fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 shutdown_connections 2023-07-14T19:37:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:33.202+0000 7f1b3520c700 1 -- 172.21.15.82:0/1496467613 wait complete. 2023-07-14T19:37:33.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:33 smithi133 ceph-mon[128894]: pgmap v281: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 372 op/s 2023-07-14T19:37:33.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:33 smithi082 ceph-mon[127461]: pgmap v281: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 372 op/s 2023-07-14T19:37:33.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:33 smithi161 ceph-mon[129184]: pgmap v281: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 372 op/s 2023-07-14T19:37:35.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:35 smithi133 ceph-mon[128894]: pgmap v282: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2023-07-14T19:37:35.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:35 smithi082 ceph-mon[127461]: pgmap v282: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2023-07-14T19:37:35.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:35 smithi161 ceph-mon[129184]: pgmap v282: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2023-07-14T19:37:37.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:37 smithi133 ceph-mon[128894]: pgmap v283: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 444 op/s 2023-07-14T19:37:37.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:37 smithi082 ceph-mon[127461]: pgmap v283: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 444 op/s 2023-07-14T19:37:37.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:37 smithi161 ceph-mon[129184]: pgmap v283: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 444 op/s 2023-07-14T19:37:39.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:39 smithi133 ceph-mon[128894]: pgmap v284: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2023-07-14T19:37:39.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:39 smithi082 ceph-mon[127461]: pgmap v284: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2023-07-14T19:37:39.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:39 smithi161 ceph-mon[129184]: pgmap v284: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2023-07-14T19:37:41.698 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:41 smithi133 ceph-mon[128894]: pgmap v285: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 420 op/s 2023-07-14T19:37:41.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:41 smithi082 ceph-mon[127461]: pgmap v285: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 420 op/s 2023-07-14T19:37:41.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:41 smithi161 ceph-mon[129184]: pgmap v285: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 420 op/s 2023-07-14T19:37:43.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:43 smithi133 ceph-mon[128894]: pgmap v286: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 435 op/s 2023-07-14T19:37:43.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:43 smithi082 ceph-mon[127461]: pgmap v286: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 435 op/s 2023-07-14T19:37:43.756 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:43 smithi161 ceph-mon[129184]: pgmap v286: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 435 op/s 2023-07-14T19:37:45.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:45 smithi133 ceph-mon[128894]: pgmap v287: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 370 op/s 2023-07-14T19:37:45.729 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:45 smithi082 ceph-mon[127461]: pgmap v287: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 370 op/s 2023-07-14T19:37:45.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:45 smithi161 ceph-mon[129184]: pgmap v287: 97 pgs: 97 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 370 op/s 2023-07-14T19:37:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 -- 172.21.15.82:0/96409999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c0ffeb0 msgr2=0x7fd43c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 --2- 172.21.15.82:0/96409999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c100280 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd4300099c0 tx=0x7fd430009cd0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 -- 172.21.15.82:0/96409999 shutdown_connections 2023-07-14T19:37:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 --2- 172.21.15.82:0/96409999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c104ef0 0x7fd43c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 --2- 172.21.15.82:0/96409999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c100280 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 --2- 172.21.15.82:0/96409999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c100850 0x7fd43c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 -- 172.21.15.82:0/96409999 >> 172.21.15.82:0/96409999 conn(0x7fd43c0fb720 msgr2=0x7fd43c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 -- 172.21.15.82:0/96409999 shutdown_connections 2023-07-14T19:37:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.318+0000 7fd44208b700 1 -- 172.21.15.82:0/96409999 wait complete. 2023-07-14T19:37:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 Processor -- start 2023-07-14T19:37:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 -- start start 2023-07-14T19:37:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c100850 0x7fd43c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c104ef0 0x7fd43c195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1961c0 con 0x7fd43c104ef0 2023-07-14T19:37:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c196330 con 0x7fd43c0ffeb0 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd44208b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c1964f0 con 0x7fd43c100850 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c104ef0 0x7fd43c195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd43b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd43b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c19bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41102/0 (socket says 172.21.15.82:41102) 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd43b7fe700 1 -- 172.21.15.82:0/3867971556 learned_addr learned my addr 172.21.15.82:0/3867971556 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.319+0000 7fd43affd700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c100850 0x7fd43c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43bfff700 1 -- 172.21.15.82:0/3867971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c100850 msgr2=0x7fd43c19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43bfff700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c100850 0x7fd43c19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43bfff700 1 -- 172.21.15.82:0/3867971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c0ffeb0 msgr2=0x7fd43c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43bfff700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43bfff700 1 -- 172.21.15.82:0/3867971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4300095f0 con 0x7fd43c104ef0 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43affd700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c100850 0x7fd43c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd43bfff700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c104ef0 0x7fd43c195b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd434007f10 tx=0x7fd43400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd438ff9700 1 -- 172.21.15.82:0/3867971556 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400ddd0 con 0x7fd43c104ef0 2023-07-14T19:37:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c1967d0 con 0x7fd43c104ef0 2023-07-14T19:37:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd438ff9700 1 -- 172.21.15.82:0/3867971556 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd43400f040 con 0x7fd43c104ef0 2023-07-14T19:37:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd438ff9700 1 -- 172.21.15.82:0/3867971556 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434005a90 con 0x7fd43c104ef0 2023-07-14T19:37:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.320+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c1a2ce0 con 0x7fd43c104ef0 2023-07-14T19:37:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.322+0000 7fd438ff9700 1 -- 172.21.15.82:0/3867971556 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd434004ad0 con 0x7fd43c104ef0 2023-07-14T19:37:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.323+0000 7fd438ff9700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fd4240a12c0 0x7fd4240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.323+0000 7fd438ff9700 1 -- 172.21.15.82:0/3867971556 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fd4340cc500 con 0x7fd43c104ef0 2023-07-14T19:37:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.323+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fd4240a12c0 0x7fd4240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.323+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fd4240a12c0 0x7fd4240a3770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4300036c0 tx=0x7fd430005fb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 Processor -- start 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 -- start start 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cb666b0 0x55c24cd10550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd0a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cc0fc50 con 0x55c24cd10f00 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cc629e0 con 0x55c24cd10a90 2023-07-14T19:37:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd44e223b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cc5be70 con 0x55c24cb666b0 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd0af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44766/0 (socket says 172.21.15.82:44766) 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 -- 172.21.15.82:0/2004175323 learned_addr learned my addr 172.21.15.82:0/2004175323 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cb666b0 0x55c24cd10550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43affd700 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd0a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 -- 172.21.15.82:0/2004175323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cb666b0 msgr2=0x55c24cd10550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cb666b0 0x55c24cd10550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.348 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 -- 172.21.15.82:0/2004175323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 msgr2=0x55c24cd0a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd0a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 -- 172.21.15.82:0/2004175323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24ca89d80 con 0x55c24cd10f00 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.325+0000 7fd43bfff700 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd0af80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd4340a1550 tx=0x7fd4340991f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801190b90224cf63 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd421ffb700 1 -- 172.21.15.82:0/2004175323 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434099630 con 0x55c24cd10f00 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd421ffb700 1 -- 172.21.15.82:0/2004175323 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd43409d660 con 0x55c24cd10f00 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd421ffb700 1 -- 172.21.15.82:0/2004175323 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434099890 con 0x55c24cd10f00 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 -- 172.21.15.82:0/2004175323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 msgr2=0x55c24cd0af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd0af80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd4340a1550 tx=0x7fd4340991f0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 -- 172.21.15.82:0/2004175323 shutdown_connections 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd0af80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cb666b0 0x55c24cd10550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 --2- 172.21.15.82:0/2004175323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd0a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 -- 172.21.15.82:0/2004175323 >> 172.21.15.82:0/2004175323 conn(0x55c24cd003d0 msgr2=0x55c24cd05530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 -- 172.21.15.82:0/2004175323 shutdown_connections 2023-07-14T19:37:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.326+0000 7fd44e223b80 1 -- 172.21.15.82:0/2004175323 wait complete. 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 -- start start 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 Processor -- start 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cb666b0 0x55c24cd19e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd1a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cbfba60 con 0x55c24cd10f00 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cbfb760 con 0x55c24cb666b0 2023-07-14T19:37:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cadf8c0 con 0x55c24cd10a90 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd44e223b80 10 client.0 _ll_register_callbacks cb 0x55c24cbf9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd43bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd14090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44772/0 (socket says 172.21.15.82:44772) 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd43bfff700 1 -- 172.21.15.82:0/3014924932 learned_addr learned my addr 172.21.15.82:0/3014924932 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cb666b0 0x55c24cd19e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.327+0000 7fd43affd700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd1a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43bfff700 1 -- 172.21.15.82:0/3014924932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 msgr2=0x55c24cd1a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43bfff700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd1a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43bfff700 1 -- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cb666b0 msgr2=0x55c24cd19e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43bfff700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cb666b0 0x55c24cd19e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43bfff700 1 -- 172.21.15.82:0/3014924932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c1a2ce0 con 0x55c24cd10f00 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43affd700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd1a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd43bfff700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd14090 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd434095b30 tx=0x7fd4340a1b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16f4ba11de1f252 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd41affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd41affd700 1 -- 172.21.15.82:0/3014924932 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434098d30 con 0x55c24cd10f00 2023-07-14T19:37:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd41affd700 1 -- 172.21.15.82:0/3014924932 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fd4340cc0d0 con 0x55c24cd10f00 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c24cbb5490 con 0x55c24cd10f00 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd41affd700 1 -- 172.21.15.82:0/3014924932 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4340cc290 con 0x55c24cd10f00 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 10 client.16257 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c24ca72c00 con 0x55c24cd10f00 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd41affd700 1 -- 172.21.15.82:0/3014924932 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c24cbb5490 con 0x55c24cd10f00 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd41affd700 1 -- 172.21.15.82:0/3014924932 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd4340a16f0 con 0x55c24cd10f00 2023-07-14T19:37:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 20 client.16257 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd23f10 0x55c24cd26320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 4 client.16257 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.328+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c24ccd9d40 con 0x55c24cd23f10 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.329+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd23f10 0x55c24cd26320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.329+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd23f10 0x55c24cd26320 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df514fffa974958f server_cookie=1cd877aa1bcf3607 in_seq=0 out_seq=0 2023-07-14T19:37:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.329+0000 7fd41affd700 10 client.16257 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd41affd700 1 -- 172.21.15.82:0/3014924932 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55c24ccd9d40 con 0x55c24cd23f10 2023-07-14T19:37:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd41affd700 10 client.16257 handle_command_reply: tid=0 2023-07-14T19:37:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 client.16257 shutdown 2023-07-14T19:37:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 msgr2=0x55c24cd14090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd14090 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd434095b30 tx=0x7fd4340a1b70 comp rx=0 tx=0).stop 2023-07-14T19:37:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 shutdown_connections 2023-07-14T19:37:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cd10f00 0x55c24cd14090 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cb666b0 0x55c24cd19e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd23f10 0x55c24cd26320 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 --2- 172.21.15.82:0/3014924932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd1a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd41affd700 0 client.16257 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 >> 172.21.15.82:0/3014924932 conn(0x55c24cd003d0 msgr2=0x55c24cd04c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 shutdown_connections 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 1 -- 172.21.15.82:0/3014924932 wait complete. 2023-07-14T19:37:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.330+0000 7fd44e223b80 20 client.16257 trim_cache size 0 max 16384 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 Processor -- start 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 -- start start 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd0fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd13fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10f00 0x55c24cd12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cd2b0e0 con 0x55c24cb666b0 2023-07-14T19:37:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cd144f0 con 0x55c24cd10a90 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd44e223b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cd0b970 con 0x55c24cd10f00 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd0fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd0fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44776/0 (socket says 172.21.15.82:44776) 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43b7fe700 1 -- 172.21.15.82:0/357067545 learned_addr learned my addr 172.21.15.82:0/357067545 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43affd700 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd13fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43b7fe700 1 -- 172.21.15.82:0/357067545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10f00 msgr2=0x55c24cd12f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10f00 0x55c24cd12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.345+0000 7fd43b7fe700 1 -- 172.21.15.82:0/357067545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 msgr2=0x55c24cd13fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd13fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd43b7fe700 1 -- 172.21.15.82:0/357067545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c24cab01f0 con 0x55c24cb666b0 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd43affd700 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd13fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd0fa40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd430009fd0 tx=0x7fd430022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a4e591b726794c server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd4227fc700 1 -- 172.21.15.82:0/357067545 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43002c040 con 0x55c24cb666b0 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd4227fc700 1 -- 172.21.15.82:0/357067545 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd43002d040 con 0x55c24cb666b0 2023-07-14T19:37:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd4227fc700 1 -- 172.21.15.82:0/357067545 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430004420 con 0x55c24cb666b0 2023-07-14T19:37:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 -- 172.21.15.82:0/357067545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 msgr2=0x55c24cd0fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd0fa40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd430009fd0 tx=0x7fd430022000 comp rx=0 tx=0).stop 2023-07-14T19:37:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 -- 172.21.15.82:0/357067545 shutdown_connections 2023-07-14T19:37:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10f00 0x55c24cd12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd0fa40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 --2- 172.21.15.82:0/357067545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10a90 0x55c24cd13fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 -- 172.21.15.82:0/357067545 >> 172.21.15.82:0/357067545 conn(0x55c24cd003d0 msgr2=0x55c24cd12880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 -- 172.21.15.82:0/357067545 shutdown_connections 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.346+0000 7fd44e223b80 1 -- 172.21.15.82:0/357067545 wait complete. 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 -- start start 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 Processor -- start 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd17ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd18530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10f00 0x55c24cd76b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cd770b0 con 0x55c24cb666b0 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cd771f0 con 0x55c24cd10f00 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c24cd77330 con 0x55c24cd10a90 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd44e223b80 10 client.0 _ll_register_callbacks cb 0x7fd42c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd43b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd17ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd43b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd17ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44782/0 (socket says 172.21.15.82:44782) 2023-07-14T19:37:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd43b7fe700 1 -- 172.21.15.82:0/974229460 learned_addr learned my addr 172.21.15.82:0/974229460 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd43bfff700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10f00 0x55c24cd76b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.347+0000 7fd43affd700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd18530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd43b7fe700 1 -- 172.21.15.82:0/974229460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 msgr2=0x55c24cd18530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd18530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd43b7fe700 1 -- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10f00 msgr2=0x55c24cd76b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10f00 0x55c24cd76b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd43b7fe700 1 -- 172.21.15.82:0/974229460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c10d300 con 0x55c24cb666b0 2023-07-14T19:37:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd43b7fe700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd17ff0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd430003a60 tx=0x7fd430033590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05baee905052027 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd41affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd41affd700 1 -- 172.21.15.82:0/974229460 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43002c050 con 0x55c24cb666b0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd41affd700 1 -- 172.21.15.82:0/974229460 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fd430033cd0 con 0x55c24cb666b0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd43409d660 con 0x55c24cb666b0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd41affd700 1 -- 172.21.15.82:0/974229460 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd430004000 con 0x55c24cb666b0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 10 client.16263 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c24ca88250 con 0x55c24cb666b0 2023-07-14T19:37:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd41affd700 1 -- 172.21.15.82:0/974229460 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd43409d660 con 0x55c24cb666b0 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd41affd700 1 -- 172.21.15.82:0/974229460 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd43004a020 con 0x55c24cb666b0 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 20 client.16263 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd785a0 0x55c24cd7a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 4 client.16263 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:37:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.348+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c24cc54e50 con 0x55c24cd785a0 2023-07-14T19:37:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.349+0000 7fd43affd700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd785a0 0x55c24cd7a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.349+0000 7fd41affd700 10 client.16263 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.349+0000 7fd43affd700 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd785a0 0x55c24cd7a9b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9c7503951dadd1 server_cookie=e4ed121afccec9c3 in_seq=0 out_seq=0 2023-07-14T19:37:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd41affd700 1 -- 172.21.15.82:0/974229460 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c24cc54e50 con 0x55c24cd785a0 2023-07-14T19:37:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd41affd700 10 client.16263 handle_command_reply: tid=0 2023-07-14T19:37:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 client.16263 shutdown 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 msgr2=0x55c24cd17ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd17ff0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd430003a60 tx=0x7fd430033590 comp rx=0 tx=0).stop 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 shutdown_connections 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c24cd10f00 0x55c24cd76b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c24cb666b0 0x55c24cd17ff0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55c24cd785a0 0x55c24cd7a9b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 --2- 172.21.15.82:0/974229460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c24cd10a90 0x55c24cd18530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd41affd700 0 client.16263 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 >> 172.21.15.82:0/974229460 conn(0x55c24cd003d0 msgr2=0x55c24cd13110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 shutdown_connections 2023-07-14T19:37:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 1 -- 172.21.15.82:0/974229460 wait complete. 2023-07-14T19:37:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.358+0000 7fd44e223b80 20 client.16263 trim_cache size 0 max 16384 2023-07-14T19:37:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:37:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:37:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:37:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:37:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fd4240a12c0 msgr2=0x7fd4240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fd4240a12c0 0x7fd4240a3770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4300036c0 tx=0x7fd430005fb0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c104ef0 msgr2=0x7fd43c195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c104ef0 0x7fd43c195b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd434007f10 tx=0x7fd43400d3b0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 shutdown_connections 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fd4240a12c0 0x7fd4240a3770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c104ef0 0x7fd43c195b50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd43c0ffeb0 0x7fd43c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 --2- 172.21.15.82:0/3867971556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd43c100850 0x7fd43c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 >> 172.21.15.82:0/3867971556 conn(0x7fd43c0fb720 msgr2=0x7fd43c0fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 shutdown_connections 2023-07-14T19:37:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:46.359+0000 7fd44208b700 1 -- 172.21.15.82:0/3867971556 wait complete. 2023-07-14T19:37:47.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:47 smithi133 ceph-mon[128894]: pgmap v288: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2023-07-14T19:37:47.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:47 smithi082 ceph-mon[127461]: pgmap v288: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2023-07-14T19:37:47.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:47 smithi161 ceph-mon[129184]: pgmap v288: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 451 op/s 2023-07-14T19:37:49.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:49 smithi133 ceph-mon[128894]: pgmap v289: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2023-07-14T19:37:49.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:49 smithi082 ceph-mon[127461]: pgmap v289: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2023-07-14T19:37:49.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:49 smithi161 ceph-mon[129184]: pgmap v289: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 361 op/s 2023-07-14T19:37:51.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:51 smithi133 ceph-mon[128894]: pgmap v290: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2023-07-14T19:37:51.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:51 smithi082 ceph-mon[127461]: pgmap v290: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2023-07-14T19:37:51.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:51 smithi161 ceph-mon[129184]: pgmap v290: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 356 op/s 2023-07-14T19:37:53.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:53 smithi133 ceph-mon[128894]: pgmap v291: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2023-07-14T19:37:53.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:53 smithi082 ceph-mon[127461]: pgmap v291: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2023-07-14T19:37:53.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:53 smithi161 ceph-mon[129184]: pgmap v291: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2023-07-14T19:37:55.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:55 smithi133 ceph-mon[128894]: pgmap v292: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 303 op/s 2023-07-14T19:37:55.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:55 smithi082 ceph-mon[127461]: pgmap v292: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 303 op/s 2023-07-14T19:37:55.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:55 smithi161 ceph-mon[129184]: pgmap v292: 97 pgs: 97 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 303 op/s 2023-07-14T19:37:57.699 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:57 smithi133 ceph-mon[128894]: pgmap v293: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2023-07-14T19:37:57.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:57 smithi082 ceph-mon[127461]: pgmap v293: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2023-07-14T19:37:57.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:57 smithi161 ceph-mon[129184]: pgmap v293: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 421 op/s 2023-07-14T19:37:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 -- 172.21.15.82:0/1087384811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94340ff230 msgr2=0x7f94340ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 --2- 172.21.15.82:0/1087384811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94340ff230 0x7f94340ff6a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f94240099c0 tx=0x7f9424009cd0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 -- 172.21.15.82:0/1087384811 shutdown_connections 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 --2- 172.21.15.82:0/1087384811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 0x7f943410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 --2- 172.21.15.82:0/1087384811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9434103ae0 0x7f9434103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 --2- 172.21.15.82:0/1087384811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94340ff230 0x7f94340ff6a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 -- 172.21.15.82:0/1087384811 >> 172.21.15.82:0/1087384811 conn(0x7f94340760c0 msgr2=0x7f94340764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 -- 172.21.15.82:0/1087384811 shutdown_connections 2023-07-14T19:37:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.470+0000 7f9438bce700 1 -- 172.21.15.82:0/1087384811 wait complete. 2023-07-14T19:37:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 Processor -- start 2023-07-14T19:37:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 -- start start 2023-07-14T19:37:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94340ff230 0x7f9434195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 0x7f9434196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 0x7f943419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419afc0 con 0x7f9434103ae0 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419b130 con 0x7f94340ffbe0 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9438bce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943419b2f0 con 0x7f94340ff230 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9432d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 0x7f943419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9432d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 0x7f943419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44800/0 (socket says 172.21.15.82:44800) 2023-07-14T19:37:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9432d9d700 1 -- 172.21.15.82:0/4148956400 learned_addr learned my addr 172.21.15.82:0/4148956400 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.471+0000 7f9431d9b700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 0x7f9434196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f943259c700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94340ff230 0x7f9434195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9431d9b700 1 -- 172.21.15.82:0/4148956400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94340ff230 msgr2=0x7f9434195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9431d9b700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94340ff230 0x7f9434195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9431d9b700 1 -- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 msgr2=0x7f943419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9431d9b700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 0x7f943419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9431d9b700 1 -- 172.21.15.82:0/4148956400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240095f0 con 0x7f94340ffbe0 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9432d9d700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 0x7f943419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9431d9b700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 0x7f9434196310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94240098e0 tx=0x7f9424009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f94237fe700 1 -- 172.21.15.82:0/4148956400 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942401d070 con 0x7f94340ffbe0 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943419b570 con 0x7f94340ffbe0 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f94237fe700 1 -- 172.21.15.82:0/4148956400 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9424005020 con 0x7f94340ffbe0 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f94237fe700 1 -- 172.21.15.82:0/4148956400 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424004a90 con 0x7f94340ffbe0 2023-07-14T19:37:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.472+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943419b800 con 0x7f94340ffbe0 2023-07-14T19:37:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.474+0000 7f94237fe700 1 -- 172.21.15.82:0/4148956400 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f94240054b0 con 0x7f94340ffbe0 2023-07-14T19:37:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.475+0000 7f94237fe700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f941c0a1340 0x7f941c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.475+0000 7f943259c700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f941c0a1340 0x7f941c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.475+0000 7f94237fe700 1 -- 172.21.15.82:0/4148956400 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f94240cd750 con 0x7f94340ffbe0 2023-07-14T19:37:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.475+0000 7f943259c700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f941c0a1340 0x7f941c0a37f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9428005950 tx=0x7f942800a300 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.475+0000 7f9444d66b80 1 Processor -- start 2023-07-14T19:37:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 -- start start 2023-07-14T19:37:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 0x55a914a26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26f00 0x55a914a20f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914925c50 con 0x55a914a26f00 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9149789e0 con 0x55a91487c6b0 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9444d66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914971e70 con 0x55a914a26a90 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f943259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 0x55a914a26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f943259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 0x55a914a26550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41152/0 (socket says 172.21.15.82:41152) 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f943259c700 1 -- 172.21.15.82:0/538210599 learned_addr learned my addr 172.21.15.82:0/538210599 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9432d9d700 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26f00 0x55a914a20f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f9431d9b700 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f943259c700 1 -- 172.21.15.82:0/538210599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 msgr2=0x55a914a209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f943259c700 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.476+0000 7f943259c700 1 -- 172.21.15.82:0/538210599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26f00 msgr2=0x55a914a20f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f943259c700 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26f00 0x55a914a20f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f943259c700 1 -- 172.21.15.82:0/538210599 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91479fd80 con 0x55a91487c6b0 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f9432d9d700 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26f00 0x55a914a20f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f943259c700 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 0x55a914a26550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f942800fd10 tx=0x7f9428012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275ce1d2e868ebab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f9420ff9700 1 -- 172.21.15.82:0/538210599 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94280107e0 con 0x55a91487c6b0 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f9420ff9700 1 -- 172.21.15.82:0/538210599 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9428020070 con 0x55a91487c6b0 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.477+0000 7f9420ff9700 1 -- 172.21.15.82:0/538210599 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428025d70 con 0x55a91487c6b0 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 -- 172.21.15.82:0/538210599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 msgr2=0x55a914a26550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 0x55a914a26550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f942800fd10 tx=0x7f9428012000 comp rx=0 tx=0).stop 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 -- 172.21.15.82:0/538210599 shutdown_connections 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26f00 0x55a914a20f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a91487c6b0 0x55a914a26550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 --2- 172.21.15.82:0/538210599 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 -- 172.21.15.82:0/538210599 >> 172.21.15.82:0/538210599 conn(0x55a914a163d0 msgr2=0x55a914a1b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 -- 172.21.15.82:0/538210599 shutdown_connections 2023-07-14T19:37:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 -- 172.21.15.82:0/538210599 wait complete. 2023-07-14T19:37:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 -- start start 2023-07-14T19:37:59.504 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.478+0000 7f9444d66b80 1 Processor -- start 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a2ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a30490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a2a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914911a60 con 0x55a914a26a90 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914911760 con 0x55a914a26f00 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9147f58c0 con 0x55a91487c6b0 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9444d66b80 10 client.0 _ll_register_callbacks cb 0x55a91490f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:59.505 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a30490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a30490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44812/0 (socket says 172.21.15.82:44812) 2023-07-14T19:37:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 -- 172.21.15.82:0/3339264462 learned_addr learned my addr 172.21.15.82:0/3339264462 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9432d9d700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a2a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f943259c700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a2ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 -- 172.21.15.82:0/3339264462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 msgr2=0x55a914a2ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a2ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 -- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 msgr2=0x55a914a2a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a2a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 -- 172.21.15.82:0/3339264462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c009190 con 0x55a914a26a90 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9431d9b700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a30490 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9424096db0 tx=0x7f94240cb400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b2db1ad7d393ab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9419ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.479+0000 7f9419ffb700 1 -- 172.21.15.82:0/3339264462 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240ba2d0 con 0x55a914a26a90 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9148cb490 con 0x55a914a26a90 2023-07-14T19:37:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9419ffb700 1 -- 172.21.15.82:0/3339264462 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f942409a8b0 con 0x55a914a26a90 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9419ffb700 1 -- 172.21.15.82:0/3339264462 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f942409e8e0 con 0x55a914a26a90 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 10 client.16275 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f942800f9f0 con 0x55a914a26a90 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9419ffb700 1 -- 172.21.15.82:0/3339264462 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9148cb490 con 0x55a914a26a90 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9419ffb700 1 -- 172.21.15.82:0/3339264462 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9424004c80 con 0x55a914a26a90 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 20 client.16275 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a3a0f0 0x55a914a3c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 4 client.16275 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9149efd40 con 0x55a914a3a0f0 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.480+0000 7f943259c700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a3a0f0 0x55a914a3c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9419ffb700 10 client.16275 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f943259c700 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a3a0f0 0x55a914a3c500 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7562a7cb6adb68 server_cookie=2758013768106c33 in_seq=0 out_seq=0 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9419ffb700 1 -- 172.21.15.82:0/3339264462 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55a9149efd40 con 0x55a914a3a0f0 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9419ffb700 10 client.16275 handle_command_reply: tid=0 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 client.16275 shutdown 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 msgr2=0x55a914a30490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a30490 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9424096db0 tx=0x7f94240cb400 comp rx=0 tx=0).stop 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 shutdown_connections 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a2a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a3a0f0 0x55a914a3c500 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a2ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9419ffb700 0 client.16275 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 --2- 172.21.15.82:0/3339264462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a30490 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 >> 172.21.15.82:0/3339264462 conn(0x55a914a163d0 msgr2=0x55a914a1ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 shutdown_connections 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.481+0000 7f9444d66b80 1 -- 172.21.15.82:0/3339264462 wait complete. 2023-07-14T19:37:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.482+0000 7f9444d66b80 20 client.16275 trim_cache size 0 max 16384 2023-07-14T19:37:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.496+0000 7f9444d66b80 1 Processor -- start 2023-07-14T19:37:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.496+0000 7f9444d66b80 1 -- start start 2023-07-14T19:37:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 0x55a914a30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a1b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9444d66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914a18450 con 0x55a91487c6b0 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9444d66b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914a41300 con 0x55a914a26f00 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9444d66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914a21970 con 0x55a914a26a90 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 0x55a914a30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 0x55a914a30450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44818/0 (socket says 172.21.15.82:44818) 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 -- 172.21.15.82:0/842405757 learned_addr learned my addr 172.21.15.82:0/842405757 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9432d9d700 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f9431d9b700 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a1b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 -- 172.21.15.82:0/842405757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 msgr2=0x55a914a1b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a1b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 -- 172.21.15.82:0/842405757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 msgr2=0x55a914a289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 -- 172.21.15.82:0/842405757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a914788c00 con 0x55a91487c6b0 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f943259c700 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 0x55a914a30450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f942800fd10 tx=0x7f9428025890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e6c45a2b30d2d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f94217fa700 1 -- 172.21.15.82:0/842405757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428025530 con 0x55a91487c6b0 2023-07-14T19:37:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f94217fa700 1 -- 172.21.15.82:0/842405757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9428014070 con 0x55a91487c6b0 2023-07-14T19:37:59.517 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.497+0000 7f94217fa700 1 -- 172.21.15.82:0/842405757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428033db0 con 0x55a91487c6b0 2023-07-14T19:37:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 -- 172.21.15.82:0/842405757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 msgr2=0x55a914a30450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 0x55a914a30450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f942800fd10 tx=0x7f9428025890 comp rx=0 tx=0).stop 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 -- 172.21.15.82:0/842405757 shutdown_connections 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a91487c6b0 0x55a914a30450 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 --2- 172.21.15.82:0/842405757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a914a26a90 0x55a914a1b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 -- 172.21.15.82:0/842405757 >> 172.21.15.82:0/842405757 conn(0x55a914a163d0 msgr2=0x55a914a2fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 -- 172.21.15.82:0/842405757 shutdown_connections 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 -- 172.21.15.82:0/842405757 wait complete. 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 -- start start 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.498+0000 7f9444d66b80 1 Processor -- start 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a36880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914a356c0 con 0x55a914a26a90 2023-07-14T19:37:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914a35830 con 0x55a914a26f00 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914a359f0 con 0x55a91487c6b0 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 10 client.0 _ll_register_callbacks cb 0x7f942409a770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a34b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44826/0 (socket says 172.21.15.82:44826) 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 -- 172.21.15.82:0/269031008 learned_addr learned my addr 172.21.15.82:0/269031008 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9432d9d700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f943259c700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a36880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.522 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 -- 172.21.15.82:0/269031008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 msgr2=0x55a914a36880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a36880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 -- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 msgr2=0x55a914a350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 -- 172.21.15.82:0/269031008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942802b040 con 0x55a914a26a90 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9431d9b700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a34b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9424096d60 tx=0x7f942409a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479979b3bba5586d server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9419ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9419ffb700 1 -- 172.21.15.82:0/269031008 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240ba2d0 con 0x55a914a26a90 2023-07-14T19:37:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9419ffb700 1 -- 172.21.15.82:0/269031008 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f942409a8b0 con 0x55a914a26a90 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9428020070 con 0x55a914a26a90 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.499+0000 7f9419ffb700 1 -- 172.21.15.82:0/269031008 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f942409e8e0 con 0x55a914a26a90 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 10 client.16281 fetch_fsmap learned FSMap version 44 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55a9147c61f0 con 0x55a914a26a90 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9419ffb700 1 -- 172.21.15.82:0/269031008 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9428020070 con 0x55a914a26a90 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9419ffb700 1 -- 172.21.15.82:0/269031008 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9424066020 con 0x55a914a26a90 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:37:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 20 client.16281 populate_metadata read hostname 'smithi082' 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a8eec0 0x55a914a912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 4 client.16281 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a91496ae50 con 0x55a914a8eec0 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f943259c700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a8eec0 0x55a914a912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f943259c700 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a8eec0 0x55a914a912d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141c313d3d590a8f server_cookie=2788f4c082d5ecff in_seq=0 out_seq=0 2023-07-14T19:37:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.500+0000 7f9419ffb700 10 client.16281 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9419ffb700 1 -- 172.21.15.82:0/269031008 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a91496ae50 con 0x55a914a8eec0 2023-07-14T19:37:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9419ffb700 10 client.16281 handle_command_reply: tid=0 2023-07-14T19:37:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 client.16281 shutdown 2023-07-14T19:37:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 msgr2=0x55a914a34b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a34b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9424096d60 tx=0x7f942409a050 comp rx=0 tx=0).stop 2023-07-14T19:37:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 shutdown_connections 2023-07-14T19:37:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a914a26f00 0x55a914a350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55a914a8eec0 0x55a914a912d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a91487c6b0 0x55a914a36880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9419ffb700 0 client.16281 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 --2- 172.21.15.82:0/269031008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a914a26a90 0x55a914a34b70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 >> 172.21.15.82:0/269031008 conn(0x55a914a163d0 msgr2=0x55a914a289e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 shutdown_connections 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 1 -- 172.21.15.82:0/269031008 wait complete. 2023-07-14T19:37:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.509+0000 7f9444d66b80 20 client.16281 trim_cache size 0 max 16384 2023-07-14T19:37:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:37:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:37:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:37:59.529 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:37:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f941c0a1340 msgr2=0x7f941c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f941c0a1340 0x7f941c0a37f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9428005950 tx=0x7f942800a300 comp rx=0 tx=0).stop 2023-07-14T19:37:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 msgr2=0x7f9434196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:37:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 0x7f9434196310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f94240098e0 tx=0x7f9424009580 comp rx=0 tx=0).stop 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 shutdown_connections 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f941c0a1340 0x7f941c0a37f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9434103ae0 0x7f943419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f94340ff230 0x7f9434195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 --2- 172.21.15.82:0/4148956400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94340ffbe0 0x7f9434196310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 >> 172.21.15.82:0/4148956400 conn(0x7f94340760c0 msgr2=0x7f94340fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 shutdown_connections 2023-07-14T19:37:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:37:59.510+0000 7f9438bce700 1 -- 172.21.15.82:0/4148956400 wait complete. 2023-07-14T19:37:59.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:37:59 smithi133 ceph-mon[128894]: pgmap v294: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 392 op/s 2023-07-14T19:37:59.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:37:59 smithi082 ceph-mon[127461]: pgmap v294: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 392 op/s 2023-07-14T19:37:59.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:37:59 smithi161 ceph-mon[129184]: pgmap v294: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 392 op/s 2023-07-14T19:38:01.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:01 smithi133 ceph-mon[128894]: pgmap v295: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2023-07-14T19:38:01.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:01 smithi082 ceph-mon[127461]: pgmap v295: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2023-07-14T19:38:01.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:01 smithi161 ceph-mon[129184]: pgmap v295: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 426 op/s 2023-07-14T19:38:03.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:03 smithi133 ceph-mon[128894]: pgmap v296: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 461 op/s 2023-07-14T19:38:03.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:03 smithi082 ceph-mon[127461]: pgmap v296: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 461 op/s 2023-07-14T19:38:03.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:03 smithi161 ceph-mon[129184]: pgmap v296: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 461 op/s 2023-07-14T19:38:05.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:05 smithi133 ceph-mon[128894]: pgmap v297: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2023-07-14T19:38:05.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:05 smithi082 ceph-mon[127461]: pgmap v297: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2023-07-14T19:38:05.757 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:05 smithi161 ceph-mon[129184]: pgmap v297: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 404 op/s 2023-07-14T19:38:07.702 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:07 smithi133 ceph-mon[128894]: pgmap v298: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 556 op/s 2023-07-14T19:38:07.702 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:07 smithi133 ceph-mon[128894]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T19:38:07.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:07 smithi082 ceph-mon[127461]: pgmap v298: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 556 op/s 2023-07-14T19:38:07.731 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:07 smithi082 ceph-mon[127461]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T19:38:07.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:07 smithi161 ceph-mon[129184]: pgmap v298: 97 pgs: 97 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 556 op/s 2023-07-14T19:38:07.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:07 smithi161 ceph-mon[129184]: osdmap e94: 12 total, 12 up, 12 in 2023-07-14T19:38:09.702 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:09 smithi133 ceph-mon[128894]: pgmap v300: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2023-07-14T19:38:09.730 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:09 smithi082 ceph-mon[127461]: pgmap v300: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2023-07-14T19:38:09.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:09 smithi161 ceph-mon[129184]: pgmap v300: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 541 op/s 2023-07-14T19:38:11.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:11 smithi133 ceph-mon[128894]: pgmap v301: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 489 op/s 2023-07-14T19:38:11.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:11 smithi082 ceph-mon[127461]: pgmap v301: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 489 op/s 2023-07-14T19:38:11.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:11 smithi161 ceph-mon[129184]: pgmap v301: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 489 op/s 2023-07-14T19:38:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 -- 172.21.15.82:0/3063926525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 msgr2=0x7fba380ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 --2- 172.21.15.82:0/3063926525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba380ff230 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b0d0 tx=0x7fba3000b3e0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 -- 172.21.15.82:0/3063926525 shutdown_connections 2023-07-14T19:38:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 --2- 172.21.15.82:0/3063926525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 0x7fba3810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 --2- 172.21.15.82:0/3063926525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba380ff230 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 --2- 172.21.15.82:0/3063926525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba380ff800 0x7fba381037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.684+0000 7fba3e69a700 1 -- 172.21.15.82:0/3063926525 >> 172.21.15.82:0/3063926525 conn(0x7fba38075d00 msgr2=0x7fba38076100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.685+0000 7fba3e69a700 1 -- 172.21.15.82:0/3063926525 shutdown_connections 2023-07-14T19:38:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.686+0000 7fba3e69a700 1 -- 172.21.15.82:0/3063926525 wait complete. 2023-07-14T19:38:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.686+0000 7fba3e69a700 1 Processor -- start 2023-07-14T19:38:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.686+0000 7fba3e69a700 1 -- start start 2023-07-14T19:38:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3e69a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba38194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3e69a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba380ff800 0x7fba38194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3e69a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 0x7fba381953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3e69a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba38199a50 con 0x7fba380fee60 2023-07-14T19:38:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3e69a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba38199bc0 con 0x7fba38103ea0 2023-07-14T19:38:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3e69a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba38199d80 con 0x7fba380ff800 2023-07-14T19:38:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba37fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba38194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba37fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba38194970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44838/0 (socket says 172.21.15.82:44838) 2023-07-14T19:38:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba37fff700 1 -- 172.21.15.82:0/223730054 learned_addr learned my addr 172.21.15.82:0/223730054 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba3cc37700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 0x7fba381953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba377fe700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba380ff800 0x7fba38194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.687+0000 7fba37fff700 1 -- 172.21.15.82:0/223730054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba380ff800 msgr2=0x7fba38194eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.688+0000 7fba37fff700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba380ff800 0x7fba38194eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.688+0000 7fba37fff700 1 -- 172.21.15.82:0/223730054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 msgr2=0x7fba381953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.688+0000 7fba37fff700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 0x7fba381953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.688+0000 7fba37fff700 1 -- 172.21.15.82:0/223730054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30009d00 con 0x7fba380fee60 2023-07-14T19:38:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.688+0000 7fba3cc37700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 0x7fba381953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.688+0000 7fba37fff700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba38194970 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba30007870 tx=0x7fba300078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.689+0000 7fba357fa700 1 -- 172.21.15.82:0/223730054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000e040 con 0x7fba380fee60 2023-07-14T19:38:12.708 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.689+0000 7fba357fa700 1 -- 172.21.15.82:0/223730054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba30009450 con 0x7fba380fee60 2023-07-14T19:38:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.689+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3819a000 con 0x7fba380fee60 2023-07-14T19:38:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.689+0000 7fba357fa700 1 -- 172.21.15.82:0/223730054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3001ba10 con 0x7fba380fee60 2023-07-14T19:38:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.689+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3819a510 con 0x7fba380fee60 2023-07-14T19:38:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.694+0000 7fba4a832b80 1 Processor -- start 2023-07-14T19:38:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.694+0000 7fba4a832b80 1 -- start start 2023-07-14T19:38:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x55860045f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba4a832b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860035ec50 con 0x55860045ff00 2023-07-14T19:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba4a832b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586003b19e0 con 0x5586002b56b0 2023-07-14T19:38:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba4a832b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586003aae70 con 0x55860045fa90 2023-07-14T19:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.695+0000 7fba357fa700 1 -- 172.21.15.82:0/223730054 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fba3001d880 con 0x7fba380fee60 2023-07-14T19:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.696+0000 7fba357fa700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fba200a1410 0x7fba200a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.696+0000 7fba357fa700 1 -- 172.21.15.82:0/223730054 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fba300cdfa0 con 0x7fba380fee60 2023-07-14T19:38:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.696+0000 7fba377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.696+0000 7fba377fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44846/0 (socket says 172.21.15.82:44846) 2023-07-14T19:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.696+0000 7fba377fe700 1 -- 172.21.15.82:0/3969381605 learned_addr learned my addr 172.21.15.82:0/3969381605 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x55860045f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fba200a1410 0x7fba200a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 -- 172.21.15.82:0/3969381605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 msgr2=0x5586004599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 -- 172.21.15.82:0/3969381605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 msgr2=0x558600459f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 -- 172.21.15.82:0/3969381605 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586001d8d80 con 0x5586002b56b0 2023-07-14T19:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.697+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x55860045f550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c024e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea59989696fee186 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.698+0000 7fba3cc37700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fba200a1410 0x7fba200a38c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba2c034000 tx=0x7fba2c032000 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.699+0000 7fba1e7fc700 1 -- 172.21.15.82:0/3969381605 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0232d0 con 0x5586002b56b0 2023-07-14T19:38:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.699+0000 7fba1e7fc700 1 -- 172.21.15.82:0/3969381605 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba2c0245d0 con 0x5586002b56b0 2023-07-14T19:38:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.699+0000 7fba1e7fc700 1 -- 172.21.15.82:0/3969381605 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c003ea0 con 0x5586002b56b0 2023-07-14T19:38:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.702+0000 7fba4a832b80 1 -- 172.21.15.82:0/3969381605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 msgr2=0x55860045f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.703+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x55860045f550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c024e70 comp rx=0 tx=0).stop 2023-07-14T19:38:12.723 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.704+0000 7fba4a832b80 1 -- 172.21.15.82:0/3969381605 shutdown_connections 2023-07-14T19:38:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.704+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.704+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x55860045f550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.704+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3969381605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.724 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.704+0000 7fba4a832b80 1 -- 172.21.15.82:0/3969381605 >> 172.21.15.82:0/3969381605 conn(0x55860044f3d0 msgr2=0x558600454530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.708+0000 7fba4a832b80 1 -- 172.21.15.82:0/3969381605 shutdown_connections 2023-07-14T19:38:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.711+0000 7fba4a832b80 1 -- 172.21.15.82:0/3969381605 wait complete. 2023-07-14T19:38:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.711+0000 7fba4a832b80 1 -- start start 2023-07-14T19:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.711+0000 7fba4a832b80 1 Processor -- start 2023-07-14T19:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x558600468ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.729 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860034aa60 con 0x55860045ff00 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860034a760 con 0x55860045fa90 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860022e8c0 con 0x5586002b56b0 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba4a832b80 10 client.0 _ll_register_callbacks cb 0x558600348700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44854/0 (socket says 172.21.15.82:44854) 2023-07-14T19:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 -- 172.21.15.82:0/1012276469 learned_addr learned my addr 172.21.15.82:0/1012276469 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba37fff700 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x558600468ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 -- 172.21.15.82:0/1012276469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 msgr2=0x558600468ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x558600468ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 -- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 msgr2=0x5586004693e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004693e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.712+0000 7fba3cc37700 1 -- 172.21.15.82:0/1012276469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c024770 con 0x55860045ff00 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba3cc37700 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004630f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fba2c023300 tx=0x7fba2c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ec37fcd4f2779a server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba177fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba177fe700 1 -- 172.21.15.82:0/1012276469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c003c10 con 0x55860045ff00 2023-07-14T19:38:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba177fe700 1 -- 172.21.15.82:0/1012276469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fba2c003d70 con 0x55860045ff00 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558600304490 con 0x55860045ff00 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba177fe700 1 -- 172.21.15.82:0/1012276469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba2c021500 con 0x55860045ff00 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba4a832b80 10 client.16293 fetch_fsmap learned FSMap version 44 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5586001c1c00 con 0x55860045ff00 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba177fe700 1 -- 172.21.15.82:0/1012276469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558600304490 con 0x55860045ff00 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.713+0000 7fba177fe700 1 -- 172.21.15.82:0/1012276469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fba2c010730 con 0x55860045ff00 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.714+0000 7fba4a832b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:38:12.734 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.714+0000 7fba4a832b80 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.714+0000 7fba4a832b80 20 client.16293 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.714+0000 7fba4a832b80 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558600473360 0x558600475770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.714+0000 7fba4a832b80 4 client.16293 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.714+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558600428d40 con 0x558600473360 2023-07-14T19:38:12.735 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.717+0000 7fba37fff700 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558600473360 0x558600475770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.728+0000 7fba177fe700 10 client.16293 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.728+0000 7fba37fff700 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558600473360 0x558600475770 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7043f3c7399fc17 server_cookie=8e9c39e12ac93bdd in_seq=0 out_seq=0 2023-07-14T19:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.733+0000 7fba177fe700 1 -- 172.21.15.82:0/1012276469 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x558600428d40 con 0x558600473360 2023-07-14T19:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.733+0000 7fba177fe700 10 client.16293 handle_command_reply: tid=0 2023-07-14T19:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.733+0000 7fba4a832b80 1 client.16293 shutdown 2023-07-14T19:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 msgr2=0x5586004630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004630f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fba2c023300 tx=0x7fba2c005720 comp rx=0 tx=0).stop 2023-07-14T19:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 shutdown_connections 2023-07-14T19:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004630f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x558600473360 0x558600475770 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x558600468ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 --2- 172.21.15.82:0/1012276469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004693e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 >> 172.21.15.82:0/1012276469 conn(0x55860044f3d0 msgr2=0x558600453c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.734+0000 7fba177fe700 0 client.16293 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.735+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 shutdown_connections 2023-07-14T19:38:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.739+0000 7fba4a832b80 1 -- 172.21.15.82:0/1012276469 wait complete. 2023-07-14T19:38:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.744+0000 7fba4a832b80 20 client.16293 trim_cache size 0 max 16384 2023-07-14T19:38:12.783 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.766+0000 7fba4a832b80 1 Processor -- start 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.766+0000 7fba4a832b80 1 -- start start 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.766+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x5586004693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba4a832b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860047a570 con 0x55860045ff00 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba4a832b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55860045a460 con 0x55860045fa90 2023-07-14T19:38:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba4a832b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x558600461650 con 0x5586002b56b0 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41210/0 (socket says 172.21.15.82:41210) 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 -- 172.21.15.82:0/4194341257 learned_addr learned my addr 172.21.15.82:0/4194341257 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba3cc37700 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 -- 172.21.15.82:0/4194341257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 msgr2=0x5586004693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x5586004693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 -- 172.21.15.82:0/4194341257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 msgr2=0x558600459f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.767+0000 7fba377fe700 1 -- 172.21.15.82:0/4194341257 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586001ff1f0 con 0x55860045fa90 2023-07-14T19:38:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.769+0000 7fba377fe700 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004599e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba28009a00 tx=0x7fba28009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0378503d9360d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.771+0000 7fba1effd700 1 -- 172.21.15.82:0/4194341257 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28011940 con 0x55860045fa90 2023-07-14T19:38:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.771+0000 7fba1effd700 1 -- 172.21.15.82:0/4194341257 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba28014440 con 0x55860045fa90 2023-07-14T19:38:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.771+0000 7fba1effd700 1 -- 172.21.15.82:0/4194341257 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28013600 con 0x55860045fa90 2023-07-14T19:38:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 -- 172.21.15.82:0/4194341257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 msgr2=0x5586004599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004599e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba28009a00 tx=0x7fba28009d10 comp rx=0 tx=0).stop 2023-07-14T19:38:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 -- 172.21.15.82:0/4194341257 shutdown_connections 2023-07-14T19:38:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x558600459f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.790 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5586002b56b0 0x5586004693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 --2- 172.21.15.82:0/4194341257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55860045fa90 0x5586004599e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.772+0000 7fba4a832b80 1 -- 172.21.15.82:0/4194341257 >> 172.21.15.82:0/4194341257 conn(0x55860044f3d0 msgr2=0x558600468c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.773+0000 7fba4a832b80 1 -- 172.21.15.82:0/4194341257 shutdown_connections 2023-07-14T19:38:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.773+0000 7fba4a832b80 1 -- 172.21.15.82:0/4194341257 wait complete. 2023-07-14T19:38:12.791 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.773+0000 7fba4a832b80 1 -- start start 2023-07-14T19:38:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.773+0000 7fba4a832b80 1 Processor -- start 2023-07-14T19:38:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x5586004c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004c0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586004c1a40 con 0x55860045ff00 2023-07-14T19:38:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586004c1bb0 con 0x5586002b56b0 2023-07-14T19:38:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586004c1d70 con 0x55860045fa90 2023-07-14T19:38:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba4a832b80 10 client.0 _ll_register_callbacks cb 0x7fba28009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004c1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44866/0 (socket says 172.21.15.82:44866) 2023-07-14T19:38:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 -- 172.21.15.82:0/3869165336 learned_addr learned my addr 172.21.15.82:0/3869165336 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba377fe700 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004c0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 -- 172.21.15.82:0/3869165336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 msgr2=0x5586004c0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004c0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 -- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 msgr2=0x5586004c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x5586004c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.774+0000 7fba3cc37700 1 -- 172.21.15.82:0/3869165336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba280096b0 con 0x55860045ff00 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.775+0000 7fba3cc37700 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004c1500 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229048a7d2e35dab server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.775+0000 7fba177fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:38:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.777+0000 7fba177fe700 1 -- 172.21.15.82:0/3869165336 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c010630 con 0x55860045ff00 2023-07-14T19:38:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.777+0000 7fba177fe700 1 -- 172.21.15.82:0/3869165336 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fba2c01d030 con 0x55860045ff00 2023-07-14T19:38:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.777+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba2c0245d0 con 0x55860045ff00 2023-07-14T19:38:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.778+0000 7fba177fe700 1 -- 172.21.15.82:0/3869165336 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba2c003680 con 0x55860045ff00 2023-07-14T19:38:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.780+0000 7fba4a832b80 10 client.16299 fetch_fsmap learned FSMap version 44 2023-07-14T19:38:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.780+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5586001d7250 con 0x55860045ff00 2023-07-14T19:38:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba177fe700 1 -- 172.21.15.82:0/3869165336 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba2c0245d0 con 0x55860045ff00 2023-07-14T19:38:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba177fe700 1 -- 172.21.15.82:0/3869165336 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 44) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fba2c02f8c0 con 0x55860045ff00 2023-07-14T19:38:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba4a832b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 44 2023-07-14T19:38:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba4a832b80 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba4a832b80 20 client.16299 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5586004d90e0 0x5586004db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba4a832b80 4 client.16299 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:38:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.782+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5586003a3e50 con 0x5586004d90e0 2023-07-14T19:38:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.784+0000 7fba37fff700 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5586004d90e0 0x5586004db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.784+0000 7fba37fff700 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5586004d90e0 0x5586004db4f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d6a66a1335b53c server_cookie=976be8067ceecc3e in_seq=0 out_seq=0 2023-07-14T19:38:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.784+0000 7fba177fe700 10 client.16299 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.878+0000 7fba177fe700 1 -- 172.21.15.82:0/3869165336 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5586003a3e50 con 0x5586004d90e0 2023-07-14T19:38:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.878+0000 7fba177fe700 10 client.16299 handle_command_reply: tid=0 2023-07-14T19:38:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.878+0000 7fba4a832b80 1 client.16299 shutdown 2023-07-14T19:38:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.878+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 msgr2=0x5586004c1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.878+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004c1500 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097a0 tx=0x7fba2c009fd0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 shutdown_connections 2023-07-14T19:38:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55860045ff00 0x5586004c1500 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586002b56b0 0x5586004c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5586004d90e0 0x5586004db4f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 --2- 172.21.15.82:0/3869165336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55860045fa90 0x5586004c0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba177fe700 0 client.16299 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 >> 172.21.15.82:0/3869165336 conn(0x55860044f3d0 msgr2=0x55860045e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 shutdown_connections 2023-07-14T19:38:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 1 -- 172.21.15.82:0/3869165336 wait complete. 2023-07-14T19:38:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.880+0000 7fba4a832b80 20 client.16299 trim_cache size 0 max 16384 2023-07-14T19:38:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:38:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:38:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:38:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:38:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fba200a1410 msgr2=0x7fba200a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fba200a1410 0x7fba200a38c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba2c034000 tx=0x7fba2c032000 comp rx=0 tx=0).stop 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 msgr2=0x7fba38194970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba38194970 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fba30007870 tx=0x7fba300078a0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 shutdown_connections 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fba200a1410 0x7fba200a38c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba38103ea0 0x7fba381953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba380fee60 0x7fba38194970 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 --2- 172.21.15.82:0/223730054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba380ff800 0x7fba38194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.881+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 >> 172.21.15.82:0/223730054 conn(0x7fba38075d00 msgr2=0x7fba381027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.882+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 shutdown_connections 2023-07-14T19:38:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:12.882+0000 7fba3e69a700 1 -- 172.21.15.82:0/223730054 wait complete. 2023-07-14T19:38:13.700 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:13 smithi133 ceph-mon[128894]: pgmap v302: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 408 op/s 2023-07-14T19:38:13.732 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:13 smithi082 ceph-mon[127461]: pgmap v302: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 408 op/s 2023-07-14T19:38:13.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:13 smithi161 ceph-mon[129184]: pgmap v302: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 408 op/s 2023-07-14T19:38:14.701 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:14 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:38:14.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:14 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:38:14.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:14 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:38:15.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:15 smithi082 ceph-mon[127461]: pgmap v303: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 379 op/s 2023-07-14T19:38:15.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:15 smithi161 ceph-mon[129184]: pgmap v303: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 379 op/s 2023-07-14T19:38:15.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:15 smithi133 ceph-mon[128894]: pgmap v303: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 379 op/s 2023-07-14T19:38:17.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:17 smithi082 ceph-mon[127461]: pgmap v304: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-07-14T19:38:17.734 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:17 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:38:17.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:17 smithi161 ceph-mon[129184]: pgmap v304: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-07-14T19:38:17.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:17 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:38:17.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:17 smithi133 ceph-mon[128894]: pgmap v304: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2023-07-14T19:38:17.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:17 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:38:18.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:18 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:38:18.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:18 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:38:18.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:18 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:38:18.950 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:18 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:38:18.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:18 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:38:18.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:18 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:38:18.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:18 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:38:18.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:18 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:38:18.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:18 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:38:19.758 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:19 smithi161 ceph-mon[129184]: pgmap v305: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-07-14T19:38:19.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:19 smithi133 ceph-mon[128894]: pgmap v305: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-07-14T19:38:19.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:19 smithi082 ceph-mon[127461]: pgmap v305: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 217 op/s 2023-07-14T19:38:21.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:21 smithi161 ceph-mon[129184]: pgmap v306: 97 pgs: 35 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 44 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 184 op/s 2023-07-14T19:38:21.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:21 smithi133 ceph-mon[128894]: pgmap v306: 97 pgs: 35 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 44 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 184 op/s 2023-07-14T19:38:21.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:21 smithi082 ceph-mon[127461]: pgmap v306: 97 pgs: 35 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 44 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 184 op/s 2023-07-14T19:38:23.759 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:23 smithi161 ceph-mon[129184]: pgmap v307: 97 pgs: 33 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 210 op/s 2023-07-14T19:38:23.955 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:23 smithi133 ceph-mon[128894]: pgmap v307: 97 pgs: 33 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 210 op/s 2023-07-14T19:38:23.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:23 smithi082 ceph-mon[127461]: pgmap v307: 97 pgs: 33 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 210 op/s 2023-07-14T19:38:25.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:25 smithi133 ceph-mon[128894]: pgmap v308: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-07-14T19:38:25.986 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:25 smithi082 ceph-mon[127461]: pgmap v308: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-07-14T19:38:26.008 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:25 smithi161 ceph-mon[129184]: pgmap v308: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 185 op/s 2023-07-14T19:38:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 -- 172.21.15.82:0/1889784809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c0ff800 msgr2=0x7fed3c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1889784809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c0ff800 0x7fed3c10b1a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b0d0 tx=0x7fed3400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 -- 172.21.15.82:0/1889784809 shutdown_connections 2023-07-14T19:38:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1889784809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c0ff800 0x7fed3c10b1a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1889784809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c1032f0 0x7fed3c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1889784809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c0ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 -- 172.21.15.82:0/1889784809 >> 172.21.15.82:0/1889784809 conn(0x7fed3c074470 msgr2=0x7fed3c074870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 -- 172.21.15.82:0/1889784809 shutdown_connections 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.058+0000 7fed41e8b700 1 -- 172.21.15.82:0/1889784809 wait complete. 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 Processor -- start 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 -- start start 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c193d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c0ff800 0x7fed3c194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c1032f0 0x7fed3c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c198f10 con 0x7fed3c1032f0 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c199080 con 0x7fed3c0fee50 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.059+0000 7fed41e8b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c199240 con 0x7fed3c0ff800 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c193d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c193d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41226/0 (socket says 172.21.15.82:41226) 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 -- 172.21.15.82:0/1994753307 learned_addr learned my addr 172.21.15.82:0/1994753307 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3affd700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c0ff800 0x7fed3c194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 -- 172.21.15.82:0/1994753307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c0ff800 msgr2=0x7fed3c194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c0ff800 0x7fed3c194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 -- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c1032f0 msgr2=0x7fed3c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c1032f0 0x7fed3c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 -- 172.21.15.82:0/1994753307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34009d00 con 0x7fed3c0fee50 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3affd700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c0ff800 0x7fed3c194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c193d20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00eab0 tx=0x7fed2c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.060+0000 7fed38ff9700 1 -- 172.21.15.82:0/1994753307 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00cad0 con 0x7fed3c0fee50 2023-07-14T19:38:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.061+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c199520 con 0x7fed3c0fee50 2023-07-14T19:38:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.061+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c1a0d60 con 0x7fed3c0fee50 2023-07-14T19:38:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.061+0000 7fed38ff9700 1 -- 172.21.15.82:0/1994753307 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed2c00cc30 con 0x7fed3c0fee50 2023-07-14T19:38:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.061+0000 7fed38ff9700 1 -- 172.21.15.82:0/1994753307 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00b5a0 con 0x7fed3c0fee50 2023-07-14T19:38:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.073+0000 7fed4e023b80 1 Processor -- start 2023-07-14T19:38:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 -- start start 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 0x55efe8c11550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c0bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8b10c50 con 0x55efe8a676b0 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8b639e0 con 0x55efe8c11a90 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.074+0000 7fed4e023b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8b5ce70 con 0x55efe8c11f00 2023-07-14T19:38:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 0x55efe8c11550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 0x55efe8c11550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44884/0 (socket says 172.21.15.82:44884) 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 -- 172.21.15.82:0/2550982392 learned_addr learned my addr 172.21.15.82:0/2550982392 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 -- 172.21.15.82:0/2550982392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 msgr2=0x55efe8c0bf80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c0bf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 -- 172.21.15.82:0/2550982392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 msgr2=0x55efe8c0b9e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0b9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.075+0000 7fed3bfff700 1 -- 172.21.15.82:0/2550982392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efe898ad80 con 0x55efe8a676b0 2023-07-14T19:38:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.076+0000 7fed3bfff700 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 0x55efe8c11550 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fed34009be0 tx=0x7fed34009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e84f666dc864fbb server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.076+0000 7fed21ffb700 1 -- 172.21.15.82:0/2550982392 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e040 con 0x55efe8a676b0 2023-07-14T19:38:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.076+0000 7fed21ffb700 1 -- 172.21.15.82:0/2550982392 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed340074e0 con 0x55efe8a676b0 2023-07-14T19:38:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.077+0000 7fed21ffb700 1 -- 172.21.15.82:0/2550982392 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3401ba90 con 0x55efe8a676b0 2023-07-14T19:38:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.077+0000 7fed4e023b80 1 -- 172.21.15.82:0/2550982392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 msgr2=0x55efe8c11550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.077+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 0x55efe8c11550 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fed34009be0 tx=0x7fed34009c10 comp rx=0 tx=0).stop 2023-07-14T19:38:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.078+0000 7fed4e023b80 1 -- 172.21.15.82:0/2550982392 shutdown_connections 2023-07-14T19:38:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.078+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c0bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.078+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8a676b0 0x55efe8c11550 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.078+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2550982392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.078+0000 7fed4e023b80 1 -- 172.21.15.82:0/2550982392 >> 172.21.15.82:0/2550982392 conn(0x55efe8c013d0 msgr2=0x55efe8c06530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.079+0000 7fed38ff9700 1 -- 172.21.15.82:0/1994753307 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fed2c00b700 con 0x7fed3c0fee50 2023-07-14T19:38:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.079+0000 7fed38ff9700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fed240a13c0 0x7fed240a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.079+0000 7fed3affd700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fed240a13c0 0x7fed240a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.079+0000 7fed38ff9700 1 -- 172.21.15.82:0/1994753307 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fed2c0cc860 con 0x7fed3c0fee50 2023-07-14T19:38:26.100 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.080+0000 7fed3affd700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fed240a13c0 0x7fed240a3870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55efe8c122d0 tx=0x7fed30009190 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.081+0000 7fed4e023b80 1 -- 172.21.15.82:0/2550982392 shutdown_connections 2023-07-14T19:38:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.082+0000 7fed4e023b80 1 -- 172.21.15.82:0/2550982392 wait complete. 2023-07-14T19:38:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.082+0000 7fed4e023b80 1 -- start start 2023-07-14T19:38:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.082+0000 7fed4e023b80 1 Processor -- start 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.082+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 0x55efe8c17d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11a90 0x55efe8c14fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c15560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8afca60 con 0x55efe8c11f00 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8afc760 con 0x55efe8a676b0 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe89e08c0 con 0x55efe8c11a90 2023-07-14T19:38:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 10 client.0 _ll_register_callbacks cb 0x55efe8afa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c15560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c15560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44892/0 (socket says 172.21.15.82:44892) 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 -- 172.21.15.82:0/3170381453 learned_addr learned my addr 172.21.15.82:0/3170381453 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3affd700 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11a90 0x55efe8c14fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 -- 172.21.15.82:0/3170381453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11a90 msgr2=0x55efe8c14fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11a90 0x55efe8c14fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 -- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 msgr2=0x55efe8c17d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 0x55efe8c17d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 -- 172.21.15.82:0/3170381453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c10ba30 con 0x55efe8c11f00 2023-07-14T19:38:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed3bfff700 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c15560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b770 tx=0x7fed3401b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d05c273b634da4 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed1affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:38:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed1affd700 1 -- 172.21.15.82:0/3170381453 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e040 con 0x55efe8c11f00 2023-07-14T19:38:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed1affd700 1 -- 172.21.15.82:0/3170381453 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fed3401b8a0 con 0x55efe8c11f00 2023-07-14T19:38:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.083+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efe8ab6490 con 0x55efe8c11f00 2023-07-14T19:38:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed1affd700 1 -- 172.21.15.82:0/3170381453 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed3401ba60 con 0x55efe8c11f00 2023-07-14T19:38:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 10 client.16308 fetch_fsmap learned FSMap version 45 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55efe8973c00 con 0x55efe8c11f00 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed1affd700 1 -- 172.21.15.82:0/3170381453 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55efe8ab6490 con 0x55efe8c11f00 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed1affd700 1 -- 172.21.15.82:0/3170381453 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 45) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fed34019040 con 0x55efe8c11f00 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 10 client.16308 fetch_fsmap finished waiting for FSMap version 45 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 20 client.16308 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8c24e90 0x55efe8c272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 4 client.16308 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:38:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.084+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efe8bdad40 con 0x55efe8c24e90 2023-07-14T19:38:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.085+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8c24e90 0x55efe8c272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.089+0000 7fed1affd700 10 client.16308 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.089+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8c24e90 0x55efe8c272a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ceca15ef2fba860 server_cookie=ebee9f3ace78ba8d in_seq=0 out_seq=0 2023-07-14T19:38:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.096+0000 7fed1affd700 1 -- 172.21.15.82:0/3170381453 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x55efe8bdad40 con 0x55efe8c24e90 2023-07-14T19:38:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.096+0000 7fed1affd700 10 client.16308 handle_command_reply: tid=0 2023-07-14T19:38:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.096+0000 7fed4e023b80 1 client.16308 shutdown 2023-07-14T19:38:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.097+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 msgr2=0x55efe8c15560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.097+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c15560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b770 tx=0x7fed3401b420 comp rx=0 tx=0).stop 2023-07-14T19:38:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 shutdown_connections 2023-07-14T19:38:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c15560 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 0x55efe8c17d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8c24e90 0x55efe8c272a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3170381453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11a90 0x55efe8c14fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 >> 172.21.15.82:0/3170381453 conn(0x55efe8c013d0 msgr2=0x55efe8c05c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.104+0000 7fed1affd700 0 client.16308 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.106+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 shutdown_connections 2023-07-14T19:38:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.106+0000 7fed4e023b80 1 -- 172.21.15.82:0/3170381453 wait complete. 2023-07-14T19:38:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.106+0000 7fed4e023b80 20 client.16308 trim_cache size 0 max 16384 2023-07-14T19:38:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 Processor -- start 2023-07-14T19:38:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 -- start start 2023-07-14T19:38:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 0x55efe8c1c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11a90 0x55efe8c12a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c28bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8c0fc50 con 0x55efe8c11a90 2023-07-14T19:38:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8c0c6f0 con 0x55efe8a676b0 2023-07-14T19:38:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.128+0000 7fed4e023b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8c162f0 con 0x55efe8c11f00 2023-07-14T19:38:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c28bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c28bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54162/0 (socket says 172.21.15.82:54162) 2023-07-14T19:38:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 -- 172.21.15.82:0/2594071613 learned_addr learned my addr 172.21.15.82:0/2594071613 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 -- 172.21.15.82:0/2594071613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 msgr2=0x55efe8c1c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 0x55efe8c1c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 -- 172.21.15.82:0/2594071613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11a90 msgr2=0x55efe8c12a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3affd700 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11a90 0x55efe8c12a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11a90 0x55efe8c12a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3bfff700 1 -- 172.21.15.82:0/2594071613 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efe89b11f0 con 0x55efe8c11f00 2023-07-14T19:38:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.129+0000 7fed3affd700 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11a90 0x55efe8c12a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:38:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.130+0000 7fed3bfff700 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c28bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed3401b5c0 tx=0x7fed34025fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e42e2f1997ab34 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.130+0000 7fed227fc700 1 -- 172.21.15.82:0/2594071613 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3400e040 con 0x55efe8c11f00 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.130+0000 7fed227fc700 1 -- 172.21.15.82:0/2594071613 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed3400bb50 con 0x55efe8c11f00 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.130+0000 7fed227fc700 1 -- 172.21.15.82:0/2594071613 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34003ea0 con 0x55efe8c11f00 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.131+0000 7fed4e023b80 1 -- 172.21.15.82:0/2594071613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 msgr2=0x55efe8c28bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.131+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c28bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed3401b5c0 tx=0x7fed34025fc0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.133+0000 7fed4e023b80 1 -- 172.21.15.82:0/2594071613 shutdown_connections 2023-07-14T19:38:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.133+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8c11f00 0x55efe8c28bb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.133+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8a676b0 0x55efe8c1c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.133+0000 7fed4e023b80 1 --2- 172.21.15.82:0/2594071613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11a90 0x55efe8c12a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.133+0000 7fed4e023b80 1 -- 172.21.15.82:0/2594071613 >> 172.21.15.82:0/2594071613 conn(0x55efe8c013d0 msgr2=0x55efe8c12300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.134+0000 7fed4e023b80 1 -- 172.21.15.82:0/2594071613 shutdown_connections 2023-07-14T19:38:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.134+0000 7fed4e023b80 1 -- 172.21.15.82:0/2594071613 wait complete. 2023-07-14T19:38:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.134+0000 7fed4e023b80 1 -- start start 2023-07-14T19:38:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.134+0000 7fed4e023b80 1 Processor -- start 2023-07-14T19:38:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 0x55efe8c0bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8c0e680 con 0x55efe8c11f00 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8c0e7f0 con 0x55efe8c11a90 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efe8c0e9b0 con 0x55efe8a676b0 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed4e023b80 10 client.0 _ll_register_callbacks cb 0x7fed2c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 0x55efe8c0bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 0x55efe8c0bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54166/0 (socket says 172.21.15.82:54166) 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 -- 172.21.15.82:0/3588528034 learned_addr learned my addr 172.21.15.82:0/3588528034 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3bfff700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3affd700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 -- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 msgr2=0x55efe8c0c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 -- 172.21.15.82:0/3588528034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 msgr2=0x55efe8c0dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c0dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3b7fe700 1 -- 172.21.15.82:0/3588528034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efe8973c00 con 0x55efe8a676b0 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.135+0000 7fed3bfff700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c0dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed3b7fe700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 0x55efe8c0bc70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fed2c096390 tx=0x7fed2c095660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e96afa693a84be server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed1affd700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed1affd700 1 -- 172.21.15.82:0/3588528034 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c099b80 con 0x55efe8a676b0 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed340074e0 con 0x55efe8a676b0 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed1affd700 1 -- 172.21.15.82:0/3588528034 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fed2c099ce0 con 0x55efe8a676b0 2023-07-14T19:38:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed1affd700 1 -- 172.21.15.82:0/3588528034 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed2c0c5020 con 0x55efe8a676b0 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 10 client.25840 fetch_fsmap learned FSMap version 45 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55efe8989250 con 0x55efe8a676b0 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed1affd700 1 -- 172.21.15.82:0/3588528034 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed340074e0 con 0x55efe8a676b0 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed1affd700 1 -- 172.21.15.82:0/3588528034 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 45) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fed2c095710 con 0x55efe8a676b0 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 45 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 20 client.25840 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8ca3e80 0x55efe8ca6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 4 client.25840 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:38:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.136+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55efe8b55e50 con 0x55efe8ca3e80 2023-07-14T19:38:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.137+0000 7fed3affd700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8ca3e80 0x55efe8ca6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.137+0000 7fed1affd700 10 client.25840 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.137+0000 7fed3affd700 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8ca3e80 0x55efe8ca6290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8176a24722d8fc server_cookie=d949826aecffe34d in_seq=0 out_seq=0 2023-07-14T19:38:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.180+0000 7fed1affd700 1 -- 172.21.15.82:0/3588528034 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55efe8b55e50 con 0x55efe8ca3e80 2023-07-14T19:38:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.180+0000 7fed1affd700 10 client.25840 handle_command_reply: tid=0 2023-07-14T19:38:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.180+0000 7fed4e023b80 1 client.25840 shutdown 2023-07-14T19:38:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 msgr2=0x55efe8c0bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 0x55efe8c0bc70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fed2c096390 tx=0x7fed2c095660 comp rx=0 tx=0).stop 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 shutdown_connections 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efe8c11f00 0x55efe8c0dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x55efe8ca3e80 0x55efe8ca6290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55efe8a676b0 0x55efe8c0bc70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 --2- 172.21.15.82:0/3588528034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efe8c11a90 0x55efe8c0c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed1affd700 0 client.25840 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 >> 172.21.15.82:0/3588528034 conn(0x55efe8c013d0 msgr2=0x55efe8c137a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 shutdown_connections 2023-07-14T19:38:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.183+0000 7fed4e023b80 1 -- 172.21.15.82:0/3588528034 wait complete. 2023-07-14T19:38:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.184+0000 7fed4e023b80 20 client.25840 trim_cache size 0 max 16384 2023-07-14T19:38:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:38:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:38:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:38:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:38:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.184+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fed240a13c0 msgr2=0x7fed240a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.184+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fed240a13c0 0x7fed240a3870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55efe8c122d0 tx=0x7fed30009190 comp rx=0 tx=0).stop 2023-07-14T19:38:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.184+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 msgr2=0x7fed3c193d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.184+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c193d20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00eab0 tx=0x7fed2c00edc0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.185+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 shutdown_connections 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.185+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fed240a13c0 0x7fed240a3870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.185+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed3c1032f0 0x7fed3c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.185+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed3c0fee50 0x7fed3c193d20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.185+0000 7fed41e8b700 1 --2- 172.21.15.82:0/1994753307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed3c0ff800 0x7fed3c194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.185+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 >> 172.21.15.82:0/1994753307 conn(0x7fed3c074470 msgr2=0x7fed3c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.186+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 shutdown_connections 2023-07-14T19:38:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:26.186+0000 7fed41e8b700 1 -- 172.21.15.82:0/1994753307 wait complete. 2023-07-14T19:38:27.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:27 smithi133 ceph-mon[128894]: pgmap v309: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-14T19:38:27.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:27 smithi082 ceph-mon[127461]: pgmap v309: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-14T19:38:28.008 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:27 smithi161 ceph-mon[129184]: pgmap v309: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2023-07-14T19:38:29.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:29 smithi133 ceph-mon[128894]: pgmap v310: 97 pgs: 23 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 65 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 210 op/s 2023-07-14T19:38:29.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:29 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:38:29.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:29 smithi082 ceph-mon[127461]: pgmap v310: 97 pgs: 23 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 65 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 210 op/s 2023-07-14T19:38:29.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:29 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:38:30.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:29 smithi161 ceph-mon[129184]: pgmap v310: 97 pgs: 23 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 65 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 210 op/s 2023-07-14T19:38:30.011 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:29 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:38:31.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:31 smithi133 ceph-mon[128894]: pgmap v311: 97 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 81 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2023-07-14T19:38:31.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:31 smithi082 ceph-mon[127461]: pgmap v311: 97 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 81 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2023-07-14T19:38:32.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:31 smithi161 ceph-mon[129184]: pgmap v311: 97 pgs: 11 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 81 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2023-07-14T19:38:33.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:33 smithi133 ceph-mon[128894]: pgmap v312: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2023-07-14T19:38:33.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:33 smithi133 ceph-mon[128894]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T19:38:33.981 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:33 smithi082 ceph-mon[127461]: pgmap v312: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2023-07-14T19:38:33.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:33 smithi082 ceph-mon[127461]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T19:38:34.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:33 smithi161 ceph-mon[129184]: pgmap v312: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 229 op/s 2023-07-14T19:38:34.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:33 smithi161 ceph-mon[129184]: osdmap e95: 12 total, 12 up, 12 in 2023-07-14T19:38:35.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:35 smithi133 ceph-mon[128894]: pgmap v314: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 201 op/s 2023-07-14T19:38:35.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:35 smithi082 ceph-mon[127461]: pgmap v314: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 201 op/s 2023-07-14T19:38:36.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:35 smithi161 ceph-mon[129184]: pgmap v314: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 201 op/s 2023-07-14T19:38:37.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:37 smithi133 ceph-mon[128894]: pgmap v315: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2023-07-14T19:38:37.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:37 smithi082 ceph-mon[127461]: pgmap v315: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2023-07-14T19:38:38.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:37 smithi161 ceph-mon[129184]: pgmap v315: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2023-07-14T19:38:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 -- 172.21.15.82:0/1893168913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b840ffe90 msgr2=0x7f5b84100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/1893168913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b840ffe90 0x7f5b84100260 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5b740099c0 tx=0x7f5b74009cd0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 -- 172.21.15.82:0/1893168913 shutdown_connections 2023-07-14T19:38:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/1893168913 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b84104ed0 0x7f5b8410caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/1893168913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b840ffe90 0x7f5b84100260 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/1893168913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b84100830 0x7f5b841047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.339+0000 7f5b8ba84700 1 -- 172.21.15.82:0/1893168913 >> 172.21.15.82:0/1893168913 conn(0x7f5b840fb720 msgr2=0x7f5b840fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.342+0000 7f5b8ba84700 1 -- 172.21.15.82:0/1893168913 shutdown_connections 2023-07-14T19:38:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.342+0000 7f5b8ba84700 1 -- 172.21.15.82:0/1893168913 wait complete. 2023-07-14T19:38:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.342+0000 7f5b8ba84700 1 Processor -- start 2023-07-14T19:38:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.342+0000 7f5b8ba84700 1 -- start start 2023-07-14T19:38:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8ba84700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b840ffe90 0x7f5b84072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8ba84700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b84100830 0x7f5b84072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8ba84700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 0x7f5b840733c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8ba84700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b841a87f0 con 0x7f5b84104ed0 2023-07-14T19:38:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8ba84700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b841a8960 con 0x7f5b840ffe90 2023-07-14T19:38:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8ba84700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84199f60 con 0x7f5b84100830 2023-07-14T19:38:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 0x7f5b840733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 0x7f5b840733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44918/0 (socket says 172.21.15.82:44918) 2023-07-14T19:38:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 -- 172.21.15.82:0/2025837070 learned_addr learned my addr 172.21.15.82:0/2025837070 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8901f700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b84100830 0x7f5b84072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b89820700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b840ffe90 0x7f5b84072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 -- 172.21.15.82:0/2025837070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b84100830 msgr2=0x7f5b84072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b84100830 0x7f5b84072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 -- 172.21.15.82:0/2025837070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b840ffe90 msgr2=0x7f5b84072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b840ffe90 0x7f5b84072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 -- 172.21.15.82:0/2025837070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740095f0 con 0x7f5b84104ed0 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b8a021700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 0x7f5b840733c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00ef30 tx=0x7f5b7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b7affd700 1 -- 172.21.15.82:0/2025837070 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c018070 con 0x7f5b84104ed0 2023-07-14T19:38:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.343+0000 7f5b7affd700 1 -- 172.21.15.82:0/2025837070 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7c00f040 con 0x7f5b84104ed0 2023-07-14T19:38:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.344+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8419a1e0 con 0x7f5b84104ed0 2023-07-14T19:38:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.344+0000 7f5b7affd700 1 -- 172.21.15.82:0/2025837070 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00cd00 con 0x7f5b84104ed0 2023-07-14T19:38:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.344+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8419a6a0 con 0x7f5b84104ed0 2023-07-14T19:38:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.346+0000 7f5b7affd700 1 -- 172.21.15.82:0/2025837070 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5b7c003a00 con 0x7f5b84104ed0 2023-07-14T19:38:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.346+0000 7f5b7affd700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5b700a12c0 0x7f5b700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.347+0000 7f5b7affd700 1 -- 172.21.15.82:0/2025837070 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f5b7c0cc770 con 0x7f5b84104ed0 2023-07-14T19:38:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.347+0000 7f5b89820700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5b700a12c0 0x7f5b700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.348+0000 7f5b89820700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5b700a12c0 0x7f5b700a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009990 tx=0x7f5b74005fb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.348+0000 7f5b97c1cb80 1 Processor -- start 2023-07-14T19:38:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.348+0000 7f5b97c1cb80 1 -- start start 2023-07-14T19:38:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564351656b0 0x55643530f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435309f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643520ec50 con 0x5564351656b0 2023-07-14T19:38:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564352619e0 con 0x55643530ff00 2023-07-14T19:38:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643525ae70 con 0x55643530fa90 2023-07-14T19:38:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435309f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353099e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54188/0 (socket says 172.21.15.82:54188) 2023-07-14T19:38:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435309f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41274/0 (socket says 172.21.15.82:41274) 2023-07-14T19:38:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 -- 172.21.15.82:0/3965528073 learned_addr learned my addr 172.21.15.82:0/3965528073 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 -- 172.21.15.82:0/3965528073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 msgr2=0x556435309f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435309f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 -- 172.21.15.82:0/3965528073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564351656b0 msgr2=0x55643530f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564351656b0 0x55643530f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8901f700 1 -- 172.21.15.82:0/3965528073 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556435088d80 con 0x55643530fa90 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.349+0000 7f5b8a021700 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435309f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.350+0000 7f5b8901f700 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353099e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009cb0 tx=0x7f5b80011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95b283ef9f54ea7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.350+0000 7f5b6ffff700 1 -- 172.21.15.82:0/3965528073 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80011d80 con 0x55643530fa90 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.350+0000 7f5b6ffff700 1 -- 172.21.15.82:0/3965528073 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b80004510 con 0x55643530fa90 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.350+0000 7f5b6ffff700 1 -- 172.21.15.82:0/3965528073 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80005220 con 0x55643530fa90 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.351+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/3965528073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 msgr2=0x5564353099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.351+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353099e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009cb0 tx=0x7f5b80011440 comp rx=0 tx=0).stop 2023-07-14T19:38:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.353+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/3965528073 shutdown_connections 2023-07-14T19:38:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.353+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435309f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.353+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564351656b0 0x55643530f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.353+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/3965528073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353099e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.353+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/3965528073 >> 172.21.15.82:0/3965528073 conn(0x5564352ff3d0 msgr2=0x556435304530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.354+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/3965528073 shutdown_connections 2023-07-14T19:38:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.354+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/3965528073 wait complete. 2023-07-14T19:38:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.354+0000 7f5b97c1cb80 1 -- start start 2023-07-14T19:38:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.354+0000 7f5b97c1cb80 1 Processor -- start 2023-07-14T19:38:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435318d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x556435319290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x556435312fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564351faa60 con 0x55643530ff00 2023-07-14T19:38:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564351fa760 con 0x5564351656b0 2023-07-14T19:38:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564350de8c0 con 0x55643530fa90 2023-07-14T19:38:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b97c1cb80 10 client.0 _ll_register_callbacks cb 0x5564351f8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b8901f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x556435319290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b8901f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x556435319290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54194/0 (socket says 172.21.15.82:54194) 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.355+0000 7f5b8901f700 1 -- 172.21.15.82:0/537285285 learned_addr learned my addr 172.21.15.82:0/537285285 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b8901f700 1 -- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 msgr2=0x556435318d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b8901f700 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435318d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b8901f700 1 -- 172.21.15.82:0/537285285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 msgr2=0x556435312fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b8901f700 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x556435312fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b8901f700 1 -- 172.21.15.82:0/537285285 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8419a6a0 con 0x55643530fa90 2023-07-14T19:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b8901f700 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x556435319290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b80004d70 tx=0x7f5b80011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f1b7bed707e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:38:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/537285285 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80004260 con 0x55643530fa90 2023-07-14T19:38:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/537285285 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f5b80003680 con 0x55643530fa90 2023-07-14T19:38:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564351b4490 con 0x55643530fa90 2023-07-14T19:38:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/537285285 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b80003840 con 0x55643530fa90 2023-07-14T19:38:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b97c1cb80 10 client.25852 fetch_fsmap learned FSMap version 46 2023-07-14T19:38:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.356+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f5b800098e0 con 0x55643530fa90 2023-07-14T19:38:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/537285285 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564351b4490 con 0x55643530fa90 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/537285285 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b80005390 con 0x55643530fa90 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b97c1cb80 10 client.25852 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b97c1cb80 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b97c1cb80 20 client.25852 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435323200 0x556435325610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b97c1cb80 4 client.25852 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:38:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564352d8d40 con 0x556435323200 2023-07-14T19:38:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.357+0000 7f5b89820700 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435323200 0x556435325610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.362+0000 7f5b6d7fa700 10 client.25852 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.362+0000 7f5b89820700 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435323200 0x556435325610 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e156e8309c0518f server_cookie=b531bf4355439b10 in_seq=0 out_seq=0 2023-07-14T19:38:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.364+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/537285285 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x5564352d8d40 con 0x556435323200 2023-07-14T19:38:39.383 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.364+0000 7f5b6d7fa700 10 client.25852 handle_command_reply: tid=0 2023-07-14T19:38:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.364+0000 7f5b97c1cb80 1 client.25852 shutdown 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 msgr2=0x556435319290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x556435319290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b80004d70 tx=0x7f5b80011c30 comp rx=0 tx=0).stop 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 shutdown_connections 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x556435312fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435318d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435323200 0x556435325610 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/537285285 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x556435319290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 >> 172.21.15.82:0/537285285 conn(0x5564352ff3d0 msgr2=0x556435303c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b6d7fa700 0 client.25852 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 shutdown_connections 2023-07-14T19:38:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.365+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/537285285 wait complete. 2023-07-14T19:38:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.366+0000 7f5b97c1cb80 20 client.25852 trim_cache size 0 max 16384 2023-07-14T19:38:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 Processor -- start 2023-07-14T19:38:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 -- start start 2023-07-14T19:38:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435312fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x55643530bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556435301480 con 0x55643530ff00 2023-07-14T19:38:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643532a3f0 con 0x5564351656b0 2023-07-14T19:38:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x556435313550 con 0x55643530fa90 2023-07-14T19:38:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x55643530bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x55643530bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44940/0 (socket says 172.21.15.82:44940) 2023-07-14T19:38:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b89820700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435312fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.389+0000 7f5b8a021700 1 -- 172.21.15.82:0/1707371008 learned_addr learned my addr 172.21.15.82:0/1707371008 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b8a021700 1 -- 172.21.15.82:0/1707371008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 msgr2=0x5564353043b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b8a021700 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b8a021700 1 -- 172.21.15.82:0/1707371008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 msgr2=0x556435312fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b8a021700 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435312fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b8a021700 1 -- 172.21.15.82:0/1707371008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556435071c00 con 0x55643530ff00 2023-07-14T19:38:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b89820700 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435312fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b8a021700 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x55643530bf80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c095e30 tx=0x7f5b7c0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c870da79b08b28b server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b78daa700 1 -- 172.21.15.82:0/1707371008 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c084040 con 0x55643530ff00 2023-07-14T19:38:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b78daa700 1 -- 172.21.15.82:0/1707371008 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b7c004ad0 con 0x55643530ff00 2023-07-14T19:38:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.390+0000 7f5b78daa700 1 -- 172.21.15.82:0/1707371008 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c099b90 con 0x55643530ff00 2023-07-14T19:38:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1707371008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 msgr2=0x55643530bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x55643530bf80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c095e30 tx=0x7f5b7c0060e0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1707371008 shutdown_connections 2023-07-14T19:38:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530ff00 0x55643530bf80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5564351656b0 0x556435312fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1707371008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55643530fa90 0x5564353043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1707371008 >> 172.21.15.82:0/1707371008 conn(0x5564352ff3d0 msgr2=0x5564353128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1707371008 shutdown_connections 2023-07-14T19:38:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1707371008 wait complete. 2023-07-14T19:38:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 -- start start 2023-07-14T19:38:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.391+0000 7f5b97c1cb80 1 Processor -- start 2023-07-14T19:38:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564351656b0 0x556435318a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530fa90 0x556435318f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435316fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556435317570 con 0x55643530fa90 2023-07-14T19:38:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564353176e0 con 0x55643530ff00 2023-07-14T19:38:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564353178a0 con 0x5564351656b0 2023-07-14T19:38:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b97c1cb80 10 client.0 _ll_register_callbacks cb 0x7f5b7c009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435316fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8a021700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435316fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41292/0 (socket says 172.21.15.82:41292) 2023-07-14T19:38:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8901f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530fa90 0x556435318f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8a021700 1 -- 172.21.15.82:0/1730551322 learned_addr learned my addr 172.21.15.82:0/1730551322 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b89820700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564351656b0 0x556435318a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8901f700 1 -- 172.21.15.82:0/1730551322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564351656b0 msgr2=0x556435318a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8901f700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564351656b0 0x556435318a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8901f700 1 -- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 msgr2=0x556435316fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8901f700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435316fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8901f700 1 -- 172.21.15.82:0/1730551322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8410d2e0 con 0x55643530fa90 2023-07-14T19:38:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b89820700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564351656b0 0x556435318a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:38:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.392+0000 7f5b8a021700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435316fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b8901f700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530fa90 0x556435318f80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x556435314120 tx=0x7f5b80004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f6b949122cd23 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b6d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:38:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/1730551322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80015030 con 0x55643530fa90 2023-07-14T19:38:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/1730551322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f5b80000e40 con 0x55643530fa90 2023-07-14T19:38:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b80004510 con 0x55643530fa90 2023-07-14T19:38:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/1730551322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b80009260 con 0x55643530fa90 2023-07-14T19:38:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 10 client.16323 fetch_fsmap learned FSMap version 46 2023-07-14T19:38:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f5b8419a1e0 con 0x55643530fa90 2023-07-14T19:38:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/1730551322 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b80004510 con 0x55643530fa90 2023-07-14T19:38:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/1730551322 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5b80004e10 con 0x55643530fa90 2023-07-14T19:38:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 10 client.16323 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T19:38:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 20 client.16323 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435377d00 0x55643537a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:39.430 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 4 client.16323 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:38:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.393+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556435253e50 con 0x556435377d00 2023-07-14T19:38:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.394+0000 7f5b89820700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435377d00 0x55643537a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.394+0000 7f5b89820700 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435377d00 0x55643537a110 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51278874b06f498 server_cookie=58b4fedcc2be2479 in_seq=0 out_seq=0 2023-07-14T19:38:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.394+0000 7f5b6d7fa700 10 client.16323 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.421+0000 7f5b6d7fa700 1 -- 172.21.15.82:0/1730551322 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556435253e50 con 0x556435377d00 2023-07-14T19:38:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.421+0000 7f5b6d7fa700 10 client.16323 handle_command_reply: tid=0 2023-07-14T19:38:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.421+0000 7f5b97c1cb80 1 client.16323 shutdown 2023-07-14T19:38:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530fa90 msgr2=0x556435318f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530fa90 0x556435318f80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x556435314120 tx=0x7f5b80004680 comp rx=0 tx=0).stop 2023-07-14T19:38:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 shutdown_connections 2023-07-14T19:38:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55643530ff00 0x556435316fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x556435377d00 0x55643537a110 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5564351656b0 0x556435318a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b6d7fa700 0 client.16323 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 --2- 172.21.15.82:0/1730551322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55643530fa90 0x556435318f80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 >> 172.21.15.82:0/1730551322 conn(0x5564352ff3d0 msgr2=0x5564353102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 shutdown_connections 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 1 -- 172.21.15.82:0/1730551322 wait complete. 2023-07-14T19:38:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b97c1cb80 20 client.16323 trim_cache size 0 max 16384 2023-07-14T19:38:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:38:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:38:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:38:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:38:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5b700a12c0 msgr2=0x7f5b700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5b700a12c0 0x7f5b700a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009990 tx=0x7f5b74005fb0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 msgr2=0x7f5b840733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.422+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 0x7f5b840733c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00ef30 tx=0x7f5b7c00c3a0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 shutdown_connections 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5b700a12c0 0x7f5b700a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b84104ed0 0x7f5b840733c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b840ffe90 0x7f5b84072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 --2- 172.21.15.82:0/2025837070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b84100830 0x7f5b84072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 >> 172.21.15.82:0/2025837070 conn(0x7f5b840fb720 msgr2=0x7f5b840fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 shutdown_connections 2023-07-14T19:38:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:39.423+0000 7f5b8ba84700 1 -- 172.21.15.82:0/2025837070 wait complete. 2023-07-14T19:38:39.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:39 smithi133 ceph-mon[128894]: pgmap v316: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2023-07-14T19:38:39.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:39 smithi082 ceph-mon[127461]: pgmap v316: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2023-07-14T19:38:40.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:39 smithi161 ceph-mon[129184]: pgmap v316: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 375 op/s 2023-07-14T19:38:41.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:41 smithi133 ceph-mon[128894]: pgmap v317: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 405 op/s 2023-07-14T19:38:41.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:41 smithi082 ceph-mon[127461]: pgmap v317: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 405 op/s 2023-07-14T19:38:42.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:41 smithi161 ceph-mon[129184]: pgmap v317: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 405 op/s 2023-07-14T19:38:43.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:43 smithi133 ceph-mon[128894]: pgmap v318: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 445 op/s 2023-07-14T19:38:43.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:43 smithi082 ceph-mon[127461]: pgmap v318: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 445 op/s 2023-07-14T19:38:44.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:43 smithi161 ceph-mon[129184]: pgmap v318: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 445 op/s 2023-07-14T19:38:45.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:45 smithi133 ceph-mon[128894]: pgmap v319: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2023-07-14T19:38:45.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:45 smithi082 ceph-mon[127461]: pgmap v319: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2023-07-14T19:38:46.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:45 smithi161 ceph-mon[129184]: pgmap v319: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2023-07-14T19:38:47.951 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:47 smithi133 ceph-mon[128894]: pgmap v320: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 477 op/s 2023-07-14T19:38:47.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:47 smithi082 ceph-mon[127461]: pgmap v320: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 477 op/s 2023-07-14T19:38:48.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:47 smithi161 ceph-mon[129184]: pgmap v320: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 477 op/s 2023-07-14T19:38:49.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:49 smithi133 ceph-mon[128894]: pgmap v321: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2023-07-14T19:38:49.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:49 smithi082 ceph-mon[127461]: pgmap v321: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2023-07-14T19:38:50.009 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:49 smithi161 ceph-mon[129184]: pgmap v321: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 403 op/s 2023-07-14T19:38:51.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:51 smithi133 ceph-mon[128894]: pgmap v322: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 412 op/s 2023-07-14T19:38:51.982 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:51 smithi082 ceph-mon[127461]: pgmap v322: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 412 op/s 2023-07-14T19:38:52.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:51 smithi161 ceph-mon[129184]: pgmap v322: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 412 op/s 2023-07-14T19:38:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 -- 172.21.15.82:0/3751830291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 msgr2=0x7f83f0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 --2- 172.21.15.82:0/3751830291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 0x7f83f0102e80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009990 tx=0x7f83e0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 -- 172.21.15.82:0/3751830291 shutdown_connections 2023-07-14T19:38:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 --2- 172.21.15.82:0/3751830291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f0103560 0x7f83f010b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 --2- 172.21.15.82:0/3751830291 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f00fe520 0x7f83f00fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 --2- 172.21.15.82:0/3751830291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 0x7f83f0102e80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.567+0000 7f83f78d6700 1 -- 172.21.15.82:0/3751830291 >> 172.21.15.82:0/3751830291 conn(0x7f83f00760c0 msgr2=0x7f83f00764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.568+0000 7f83f78d6700 1 -- 172.21.15.82:0/3751830291 shutdown_connections 2023-07-14T19:38:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.568+0000 7f83f78d6700 1 -- 172.21.15.82:0/3751830291 wait complete. 2023-07-14T19:38:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.568+0000 7f83f78d6700 1 Processor -- start 2023-07-14T19:38:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.568+0000 7f83f78d6700 1 -- start start 2023-07-14T19:38:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f78d6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f00fe520 0x7f83f0071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f78d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 0x7f83f0072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f78d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 0x7f83f006ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f78d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f006b350 con 0x7f83f00feec0 2023-07-14T19:38:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f78d6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f006b4c0 con 0x7f83f00fe520 2023-07-14T19:38:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f78d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f006b680 con 0x7f83f0103560 2023-07-14T19:38:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 0x7f83f006ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.596 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 0x7f83f006ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54222/0 (socket says 172.21.15.82:54222) 2023-07-14T19:38:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 -- 172.21.15.82:0/1781807952 learned_addr learned my addr 172.21.15.82:0/1781807952 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 -- 172.21.15.82:0/1781807952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f00fe520 msgr2=0x7f83f0071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f00fe520 0x7f83f0071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 -- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 msgr2=0x7f83f0072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 0x7f83f0072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 -- 172.21.15.82:0/1781807952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00095f0 con 0x7f83f0103560 2023-07-14T19:38:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.569+0000 7f83f5e73700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 0x7f83f006ab80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83e8007ee0 tx=0x7f83e800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.570+0000 7f83e67fc700 1 -- 172.21.15.82:0/1781807952 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8017070 con 0x7f83f0103560 2023-07-14T19:38:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.570+0000 7f83e67fc700 1 -- 172.21.15.82:0/1781807952 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83e800f040 con 0x7f83f0103560 2023-07-14T19:38:52.599 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.570+0000 7f83e67fc700 1 -- 172.21.15.82:0/1781807952 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8005740 con 0x7f83f0103560 2023-07-14T19:38:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.570+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f006f810 con 0x7f83f0103560 2023-07-14T19:38:52.600 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.570+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f006fcd0 con 0x7f83f0103560 2023-07-14T19:38:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.576+0000 7f8403a6eb80 1 Processor -- start 2023-07-14T19:38:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f83e67fc700 1 -- 172.21.15.82:0/1781807952 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f83e80058a0 con 0x7f83f0103560 2023-07-14T19:38:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f8403a6eb80 1 -- start start 2023-07-14T19:38:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595186006b0 0x5595187aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187aaea0 0x5595187a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595186a9c50 con 0x5595187aaa30 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.578+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595186fc9e0 con 0x5595187aaea0 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595186f5e70 con 0x5595186006b0 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595186006b0 0x5595187aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595186006b0 0x5595187aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54226/0 (socket says 172.21.15.82:54226) 2023-07-14T19:38:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 -- 172.21.15.82:0/1974941801 learned_addr learned my addr 172.21.15.82:0/1974941801 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83e67fc700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f83dc0a1340 0x7f83dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187aaea0 0x5595187a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f5e73700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f83dc0a1340 0x7f83dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83e67fc700 1 -- 172.21.15.82:0/1781807952 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f83e80cbf60 con 0x7f83f0103560 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 -- 172.21.15.82:0/1974941801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595186006b0 msgr2=0x5595187aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595186006b0 0x5595187aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 -- 172.21.15.82:0/1974941801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 msgr2=0x5595187a49e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-14T19:38:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187a49e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f4e71700 1 -- 172.21.15.82:0/1974941801 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559518523d80 con 0x5595187aaea0 2023-07-14T19:38:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.579+0000 7f83f5e73700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f83dc0a1340 0x7f83dc0a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83e80c9d10 tx=0x7f83e8016890 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.581+0000 7f83f4e71700 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187aaea0 0x5595187a4f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83e00098e0 tx=0x7f83e0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f57b7e0d2879c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.582+0000 7f83db7fe700 1 -- 172.21.15.82:0/1974941801 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000b9d0 con 0x5595187aaea0 2023-07-14T19:38:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.582+0000 7f83db7fe700 1 -- 172.21.15.82:0/1974941801 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83e003e070 con 0x5595187aaea0 2023-07-14T19:38:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.582+0000 7f83db7fe700 1 -- 172.21.15.82:0/1974941801 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e00056e0 con 0x5595187aaea0 2023-07-14T19:38:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.583+0000 7f8403a6eb80 1 -- 172.21.15.82:0/1974941801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187aaea0 msgr2=0x5595187a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.583+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187aaea0 0x5595187a4f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83e00098e0 tx=0x7f83e0000bc0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.584+0000 7f8403a6eb80 1 -- 172.21.15.82:0/1974941801 shutdown_connections 2023-07-14T19:38:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.584+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187aaea0 0x5595187a4f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.584+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595186006b0 0x5595187aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.584+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/1974941801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.584+0000 7f8403a6eb80 1 -- 172.21.15.82:0/1974941801 >> 172.21.15.82:0/1974941801 conn(0x55951879a3d0 msgr2=0x55951879f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.585+0000 7f8403a6eb80 1 -- 172.21.15.82:0/1974941801 shutdown_connections 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.585+0000 7f8403a6eb80 1 -- 172.21.15.82:0/1974941801 wait complete. 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.585+0000 7f8403a6eb80 1 -- start start 2023-07-14T19:38:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.585+0000 7f8403a6eb80 1 Processor -- start 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x5595187b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518695a60 con 0x5595187aaa30 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518695760 con 0x5595186006b0 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595185798c0 con 0x5595187aaea0 2023-07-14T19:38:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f8403a6eb80 10 client.0 _ll_register_callbacks cb 0x559518693700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f5672700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f5672700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44970/0 (socket says 172.21.15.82:44970) 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f5672700 1 -- 172.21.15.82:0/2754297433 learned_addr learned my addr 172.21.15.82:0/2754297433 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f4e71700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x5595187b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f4e71700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f5672700 1 -- 172.21.15.82:0/2754297433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 msgr2=0x5595187ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.586+0000 7f83f5672700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83f5672700 1 -- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 msgr2=0x5595187b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83f5672700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x5595187b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83f5672700 1 -- 172.21.15.82:0/2754297433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f0067520 con 0x5595187aaa30 2023-07-14T19:38:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83f5672700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187b4370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00bcd0 tx=0x7f83ec00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e978153bef9798f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-07-14T19:38:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83d8ff9700 1 -- 172.21.15.82:0/2754297433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00dc20 con 0x5595187aaa30 2023-07-14T19:38:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83d8ff9700 1 -- 172.21.15.82:0/2754297433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f83ec00dd80 con 0x5595187aaa30 2023-07-14T19:38:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55951864f490 con 0x5595187aaa30 2023-07-14T19:38:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83d8ff9700 1 -- 172.21.15.82:0/2754297433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83ec004830 con 0x5595187aaa30 2023-07-14T19:38:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 10 client.16329 fetch_fsmap learned FSMap version 46 2023-07-14T19:38:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55951850cc00 con 0x5595187aaa30 2023-07-14T19:38:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83d8ff9700 1 -- 172.21.15.82:0/2754297433 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55951864f490 con 0x5595187aaa30 2023-07-14T19:38:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f83d8ff9700 1 -- 172.21.15.82:0/2754297433 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f83ec010460 con 0x5595187aaa30 2023-07-14T19:38:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 10 client.16329 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T19:38:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 20 client.16329 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595187be3a0 0x5595187c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 4 client.16329 mds_command: new command op to 24524 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-14T19:38:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.587+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559518773d40 con 0x5595187be3a0 2023-07-14T19:38:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.588+0000 7f83f4e71700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595187be3a0 0x5595187c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.588+0000 7f83d8ff9700 10 client.16329 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.588+0000 7f83f4e71700 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595187be3a0 0x5595187c07b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0ce062908be71f server_cookie=6119e8b9d395ddf7 in_seq=0 out_seq=0 2023-07-14T19:38:52.618 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.589+0000 7f83d8ff9700 1 -- 172.21.15.82:0/2754297433 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18017 (crc 0 0 0) 0x559518773d40 con 0x5595187be3a0 2023-07-14T19:38:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.589+0000 7f83d8ff9700 10 client.16329 handle_command_reply: tid=0 2023-07-14T19:38:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.589+0000 7f8403a6eb80 1 client.16329 shutdown 2023-07-14T19:38:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.589+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 msgr2=0x5595187b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.589+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187b4370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00bcd0 tx=0x7f83ec00d2e0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 shutdown_connections 2023-07-14T19:38:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595187be3a0 0x5595187c07b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.620 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x5595187b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/2754297433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187aaa30 0x5595187b4370 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 >> 172.21.15.82:0/2754297433 conn(0x55951879a3d0 msgr2=0x55951879f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f83d8ff9700 0 client.16329 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 shutdown_connections 2023-07-14T19:38:52.621 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.590+0000 7f8403a6eb80 1 -- 172.21.15.82:0/2754297433 wait complete. 2023-07-14T19:38:52.622 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.591+0000 7f8403a6eb80 20 client.16329 trim_cache size 0 max 16384 2023-07-14T19:38:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 Processor -- start 2023-07-14T19:38:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 -- start start 2023-07-14T19:38:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595186006b0 0x55951879f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187b3f40 0x5595187a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595187a5620 con 0x5595186006b0 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595187a9f30 con 0x5595187b3f40 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595187a4e30 con 0x5595187aaea0 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f83f5672700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f83f5672700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.82:54242/0 (socket says 172.21.15.82:54242) 2023-07-14T19:38:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f83f5672700 1 -- 172.21.15.82:0/135649836 learned_addr learned my addr 172.21.15.82:0/135649836 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.613+0000 7f83f4e71700 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187b3f40 0x5595187a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f5672700 1 -- 172.21.15.82:0/135649836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187b3f40 msgr2=0x5595187a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f5672700 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187b3f40 0x5595187a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f5672700 1 -- 172.21.15.82:0/135649836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595186006b0 msgr2=0x55951879f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f5672700 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595186006b0 0x55951879f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f5672700 1 -- 172.21.15.82:0/135649836 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55951854a1f0 con 0x5595187aaea0 2023-07-14T19:38:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f4e71700 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187b3f40 0x5595187a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:38:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83f5672700 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187b3a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f83ec010840 tx=0x7f83ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c422382534a2835e server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83dbfff700 1 -- 172.21.15.82:0/135649836 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00d710 con 0x5595187aaea0 2023-07-14T19:38:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83dbfff700 1 -- 172.21.15.82:0/135649836 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ec010ab0 con 0x5595187aaea0 2023-07-14T19:38:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.614+0000 7f83dbfff700 1 -- 172.21.15.82:0/135649836 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec003970 con 0x5595187aaea0 2023-07-14T19:38:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 -- 172.21.15.82:0/135649836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 msgr2=0x5595187b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187b3a00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f83ec010840 tx=0x7f83ec000bc0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 -- 172.21.15.82:0/135649836 shutdown_connections 2023-07-14T19:38:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595187b3f40 0x5595187a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595186006b0 0x55951879f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/135649836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x5595187b3a00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 -- 172.21.15.82:0/135649836 >> 172.21.15.82:0/135649836 conn(0x55951879a3d0 msgr2=0x5595187a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.615+0000 7f8403a6eb80 1 -- 172.21.15.82:0/135649836 shutdown_connections 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.616+0000 7f8403a6eb80 1 -- 172.21.15.82:0/135649836 wait complete. 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.616+0000 7f8403a6eb80 1 -- start start 2023-07-14T19:38:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.616+0000 7f8403a6eb80 1 Processor -- start 2023-07-14T19:38:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.616+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x55951880dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x55951880e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f8403a6eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187b3f40 0x559518807f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518808520 con 0x5595187b3f40 2023-07-14T19:38:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518808690 con 0x5595186006b0 2023-07-14T19:38:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f8403a6eb80 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x559518808850 con 0x5595187aaea0 2023-07-14T19:38:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f8403a6eb80 10 client.0 _ll_register_callbacks cb 0x7f83ec00b750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-14T19:38:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f4e71700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x55951880dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x55951880e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f4e71700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x55951880dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41330/0 (socket says 172.21.15.82:41330) 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f4e71700 1 -- 172.21.15.82:0/506882421 learned_addr learned my addr 172.21.15.82:0/506882421 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 -- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 msgr2=0x55951880dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x55951880dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 -- 172.21.15.82:0/506882421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187b3f40 msgr2=0x559518807f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187b3f40 0x559518807f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 -- 172.21.15.82:0/506882421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559518523d80 con 0x5595187aaea0 2023-07-14T19:38:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83f5672700 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x55951880e0e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0107c0 tx=0x7f83ec00de70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b27da1eb622f14da server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.617+0000 7f83d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f83d8ff9700 1 -- 172.21.15.82:0/506882421 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec004220 con 0x5595187aaea0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f83d8ff9700 1 -- 172.21.15.82:0/506882421 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f83ec004380 con 0x5595187aaea0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83e003e070 con 0x5595187aaea0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f83d8ff9700 1 -- 172.21.15.82:0/506882421 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83ec00d710 con 0x5595187aaea0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f8403a6eb80 10 client.25873 fetch_fsmap learned FSMap version 46 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55951850cc00 con 0x5595187aaea0 2023-07-14T19:38:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.618+0000 7f83d8ff9700 1 -- 172.21.15.82:0/506882421 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83ec03b080 con 0x5595187aaea0 2023-07-14T19:38:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f83d8ff9700 1 -- 172.21.15.82:0/506882421 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 46) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f83ec003680 con 0x5595187aaea0 2023-07-14T19:38:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f8403a6eb80 10 client.25873 fetch_fsmap finished waiting for FSMap version 46 2023-07-14T19:38:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f8403a6eb80 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-14T19:38:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f8403a6eb80 20 client.25873 populate_metadata read hostname 'smithi082' 2023-07-14T19:38:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595188156c0 0x559518817aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f8403a6eb80 4 client.25873 mds_command: new command op to 24524 tid=0[{"prefix": "flush journal"}] 2023-07-14T19:38:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.619+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 --> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5595186eee50 con 0x5595188156c0 2023-07-14T19:38:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.620+0000 7f83f4e71700 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595188156c0 0x559518817aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.620+0000 7f83f4e71700 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595188156c0 0x559518817aa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d5b82da30a52ed server_cookie=9f175c0ef0deb811 in_seq=0 out_seq=0 2023-07-14T19:38:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.620+0000 7f83d8ff9700 10 client.25873 ms_handle_connect on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:52.660 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.640+0000 7f83d8ff9700 1 -- 172.21.15.82:0/506882421 <== mds.0 v2:172.21.15.133:6838/242970631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5595186eee50 con 0x5595188156c0 2023-07-14T19:38:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.640+0000 7f83d8ff9700 10 client.25873 handle_command_reply: tid=0 2023-07-14T19:38:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 client.25873 shutdown 2023-07-14T19:38:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 msgr2=0x55951880e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x55951880e0e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0107c0 tx=0x7f83ec00de70 comp rx=0 tx=0).stop 2023-07-14T19:38:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 shutdown_connections 2023-07-14T19:38:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5595187b3f40 0x559518807f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595186006b0 0x55951880dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.133:6838/242970631,v1:172.21.15.133:6839/242970631] conn(0x5595188156c0 0x559518817aa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 --2- 172.21.15.82:0/506882421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5595187aaea0 0x55951880e0e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 >> 172.21.15.82:0/506882421 conn(0x55951879a3d0 msgr2=0x5595187a85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f83d8ff9700 0 client.25873 ms_handle_reset on v2:172.21.15.133:6838/242970631 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 shutdown_connections 2023-07-14T19:38:52.662 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.641+0000 7f8403a6eb80 1 -- 172.21.15.82:0/506882421 wait complete. 2023-07-14T19:38:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f8403a6eb80 20 client.25873 trim_cache size 0 max 16384 2023-07-14T19:38:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:{ 2023-07-14T19:38:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "message": "", 2023-07-14T19:38:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout: "return_code": 0 2023-07-14T19:38:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stdout:} 2023-07-14T19:38:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f83dc0a1340 msgr2=0x7f83dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f83dc0a1340 0x7f83dc0a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83e80c9d10 tx=0x7f83e8016890 comp rx=0 tx=0).stop 2023-07-14T19:38:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 msgr2=0x7f83f006ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 0x7f83f006ab80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83e8007ee0 tx=0x7f83e800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 shutdown_connections 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f83dc0a1340 0x7f83dc0a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f0103560 0x7f83f006ab80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f00fe520 0x7f83f0071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 --2- 172.21.15.82:0/1781807952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00feec0 0x7f83f0072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.642+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 >> 172.21.15.82:0/1781807952 conn(0x7f83f00760c0 msgr2=0x7f83f00fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.644+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 shutdown_connections 2023-07-14T19:38:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi082.front.sepia.ceph.com.smithi082.stderr:2023-07-14T19:38:52.644+0000 7f83f78d6700 1 -- 172.21.15.82:0/1781807952 wait complete. 2023-07-14T19:38:53.055 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-07-14T19:38:53.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-14T19:38:53.413 DEBUG:teuthology.parallel:result is None 2023-07-14T19:38:53.414 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-14T19:38:53.508 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-14T19:38:53.508 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-14T19:38:53.519 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:38:53.527 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-14T19:38:53.557 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-14T19:38:53.557 INFO:teuthology.task.exec:Executing custom commands... 2023-07-14T19:38:53.558 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:38:53.558 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-14T19:38:53.597 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-14T19:38:53.664 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:53 smithi082 ceph-mon[127461]: pgmap v323: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2023-07-14T19:38:53.666 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-14T19:38:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.809+0000 7f76fb848700 1 -- 172.21.15.82:0/4184253807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f40ff420 msgr2=0x7f76f40ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.809+0000 7f76fb848700 1 --2- 172.21.15.82:0/4184253807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f40ff420 0x7f76f40ff890 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009990 tx=0x7f76e4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:38:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.809+0000 7f76fb848700 1 -- 172.21.15.82:0/4184253807 shutdown_connections 2023-07-14T19:38:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 --2- 172.21.15.82:0/4184253807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f410a270 0x7f76f410c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:53.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 --2- 172.21.15.82:0/4184253807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76f4076ab0 0x7f76f40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:53.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 --2- 172.21.15.82:0/4184253807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f40ff420 0x7f76f40ff890 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:53.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- 172.21.15.82:0/4184253807 >> 172.21.15.82:0/4184253807 conn(0x7f76f40753a0 msgr2=0x7f76f40757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:53.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- 172.21.15.82:0/4184253807 shutdown_connections 2023-07-14T19:38:53.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- 172.21.15.82:0/4184253807 wait complete. 2023-07-14T19:38:53.830 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 Processor -- start 2023-07-14T19:38:53.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- start start 2023-07-14T19:38:53.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 0x7f76f4195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f40ff420 0x7f76f41962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76f410a270 0x7f76f419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419af60 con 0x7f76f4076ab0 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419b0d0 con 0x7f76f40ff420 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76fb848700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419b290 con 0x7f76f410a270 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.810+0000 7f76f95e4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 0x7f76f4195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 0x7f76f4195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44996/0 (socket says 172.21.15.82:44996) 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 -- 172.21.15.82:0/3543239668 learned_addr learned my addr 172.21.15.82:0/3543239668 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f8de3700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f40ff420 0x7f76f41962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f9de5700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76f410a270 0x7f76f419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 -- 172.21.15.82:0/3543239668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76f410a270 msgr2=0x7f76f419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76f410a270 0x7f76f419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 -- 172.21.15.82:0/3543239668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f40ff420 msgr2=0x7f76f41962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:53.834 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f40ff420 0x7f76f41962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:53.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 -- 172.21.15.82:0/3543239668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e40095f0 con 0x7f76f4076ab0 2023-07-14T19:38:53.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76f95e4700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 0x7f76f4195d70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f76f000b530 tx=0x7f76f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:53.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011830 con 0x7f76f4076ab0 2023-07-14T19:38:53.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f419b570 con 0x7f76f4076ab0 2023-07-14T19:38:53.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f0011e50 con 0x7f76f4076ab0 2023-07-14T19:38:53.835 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f000f2e0 con 0x7f76f4076ab0 2023-07-14T19:38:53.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.811+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f41a2db0 con 0x7f76f4076ab0 2023-07-14T19:38:53.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.812+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f4105030 con 0x7f76f4076ab0 2023-07-14T19:38:53.836 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.813+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f76f0010380 con 0x7f76f4076ab0 2023-07-14T19:38:53.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.814+0000 7f76ea7fc700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f76e00a12c0 0x7f76e00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:53.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.814+0000 7f76f8de3700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f76e00a12c0 0x7f76e00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:53.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.814+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f76f00ccce0 con 0x7f76f4076ab0 2023-07-14T19:38:53.837 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.814+0000 7f76f8de3700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f76e00a12c0 0x7f76e00a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76f4197590 tx=0x7f76e4005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:53.838 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:53.817+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f76f0096200 con 0x7f76f4076ab0 2023-07-14T19:38:53.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:53 smithi133 ceph-mon[128894]: pgmap v323: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2023-07-14T19:38:54.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:53 smithi161 ceph-mon[129184]: pgmap v323: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2023-07-14T19:38:54.049 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.030+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 --> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f76f4197340 con 0x7f76e00a12c0 2023-07-14T19:38:54.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.038+0000 7f76ea7fc700 1 -- 172.21.15.82:0/3543239668 <== mgr.15414 v2:172.21.15.82:6800/2084410629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f76f4197340 con 0x7f76e00a12c0 2023-07-14T19:38:54.057 INFO:teuthology.orchestra.run.smithi082.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-14T00:00:00", "created": "2023-07-14T19:28:34", "first": "2023-07-14T19:29:01", "last": "2023-07-14T19:38:00", "last_pruned": "2023-07-14T19:38:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-07-14T19:38:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f76e00a12c0 msgr2=0x7f76e00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f76e00a12c0 0x7f76e00a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76f4197590 tx=0x7f76e4005fd0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 msgr2=0x7f76f4195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 0x7f76f4195d70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f76f000b530 tx=0x7f76f000b840 comp rx=0 tx=0).stop 2023-07-14T19:38:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 shutdown_connections 2023-07-14T19:38:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f76e00a12c0 0x7f76e00a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76f410a270 0x7f76f419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76f4076ab0 0x7f76f4195d70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 --2- 172.21.15.82:0/3543239668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f40ff420 0x7f76f41962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 >> 172.21.15.82:0/3543239668 conn(0x7f76f40753a0 msgr2=0x7f76f4102170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 shutdown_connections 2023-07-14T19:38:54.061 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.040+0000 7f76fb848700 1 -- 172.21.15.82:0/3543239668 wait complete. 2023-07-14T19:38:54.071 DEBUG:teuthology.orchestra.run.smithi082:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-14T19:38:54.194 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 -- 172.21.15.82:0/3330917947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b48103e60 msgr2=0x7f7b4810ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 --2- 172.21.15.82:0/3330917947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b48103e60 0x7f7b4810ba30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000b0d0 tx=0x7f7b4000b3e0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 -- 172.21.15.82:0/3330917947 shutdown_connections 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 --2- 172.21.15.82:0/3330917947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b48103e60 0x7f7b4810ba30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 --2- 172.21.15.82:0/3330917947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b480fee20 0x7f7b480ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 --2- 172.21.15.82:0/3330917947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b480ff7c0 0x7f7b48103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 -- 172.21.15.82:0/3330917947 >> 172.21.15.82:0/3330917947 conn(0x7f7b48076ae0 msgr2=0x7f7b48076ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 -- 172.21.15.82:0/3330917947 shutdown_connections 2023-07-14T19:38:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.175+0000 7f7b5007e700 1 -- 172.21.15.82:0/3330917947 wait complete. 2023-07-14T19:38:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 Processor -- start 2023-07-14T19:38:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 -- start start 2023-07-14T19:38:54.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 0x7f7b48100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b480ff7c0 0x7f7b48100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b48103e60 0x7f7b481a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.198 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b481a4560 con 0x7f7b480fee20 2023-07-14T19:38:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b481a46a0 con 0x7f7b48103e60 2023-07-14T19:38:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b481a4810 con 0x7f7b480ff7c0 2023-07-14T19:38:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 0x7f7b48100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 0x7f7b48100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45008/0 (socket says 172.21.15.82:45008) 2023-07-14T19:38:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 -- 172.21.15.82:0/556018648 learned_addr learned my addr 172.21.15.82:0/556018648 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4d619700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b480ff7c0 0x7f7b48100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4e61b700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b48103e60 0x7f7b481a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 -- 172.21.15.82:0/556018648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b480ff7c0 msgr2=0x7f7b48100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b480ff7c0 0x7f7b48100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 -- 172.21.15.82:0/556018648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b48103e60 msgr2=0x7f7b481a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b48103e60 0x7f7b481a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 -- 172.21.15.82:0/556018648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40009d00 con 0x7f7b480fee20 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b4de1a700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 0x7f7b48100a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800d950 tx=0x7f7b3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800f890 con 0x7f7b480fee20 2023-07-14T19:38:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b38010460 con 0x7f7b480fee20 2023-07-14T19:38:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800e610 con 0x7f7b480fee20 2023-07-14T19:38:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.176+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b481a4af0 con 0x7f7b480fee20 2023-07-14T19:38:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.177+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b481a4fb0 con 0x7f7b480fee20 2023-07-14T19:38:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.177+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b48078600 con 0x7f7b480fee20 2023-07-14T19:38:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.178+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7b380108d0 con 0x7f7b480fee20 2023-07-14T19:38:54.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.179+0000 7f7b3effd700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f7b340a12c0 0x7f7b340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.179+0000 7f7b4d619700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f7b340a12c0 0x7f7b340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.179+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f7b380cca70 con 0x7f7b480fee20 2023-07-14T19:38:54.202 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.179+0000 7f7b4d619700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f7b340a12c0 0x7f7b340a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b48102220 tx=0x7f7b44009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:54.203 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.183+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f7b38095ee0 con 0x7f7b480fee20 2023-07-14T19:38:54.415 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.396+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 --> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7b48068cc0 con 0x7f7b340a12c0 2023-07-14T19:38:54.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.404+0000 7f7b3effd700 1 -- 172.21.15.82:0/556018648 <== mgr.15414 v2:172.21.15.82:6800/2084410629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f7b48068cc0 con 0x7f7b340a12c0 2023-07-14T19:38:54.423 INFO:teuthology.orchestra.run.smithi082.stdout:/ 1M 6M3h 2023-07-14T19:38:54.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.406+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f7b340a12c0 msgr2=0x7f7b340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.406+0000 7f7b5007e700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f7b340a12c0 0x7f7b340a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b48102220 tx=0x7f7b44009260 comp rx=0 tx=0).stop 2023-07-14T19:38:54.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.406+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 msgr2=0x7f7b48100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.406+0000 7f7b5007e700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 0x7f7b48100a00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800d950 tx=0x7f7b3800dc60 comp rx=0 tx=0).stop 2023-07-14T19:38:54.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 shutdown_connections 2023-07-14T19:38:54.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f7b340a12c0 0x7f7b340a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b48103e60 0x7f7b481a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b480fee20 0x7f7b48100a00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 --2- 172.21.15.82:0/556018648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b480ff7c0 0x7f7b48100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 >> 172.21.15.82:0/556018648 conn(0x7f7b48076ae0 msgr2=0x7f7b480685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:54.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 shutdown_connections 2023-07-14T19:38:54.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.407+0000 7f7b5007e700 1 -- 172.21.15.82:0/556018648 wait complete. 2023-07-14T19:38:54.438 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-14T19:38:54.449 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-14T19:38:54.449 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81c9387d2e7576da065747ec9c16d1c7d69016ec/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-14T19:38:54.449 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-14T19:38:54.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:38:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.594+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/3109839283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040feb00 msgr2=0x7f0e040fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.594+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/3109839283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040feb00 0x7f0e040fef70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009990 tx=0x7f0df4009ca0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/3109839283 shutdown_connections 2023-07-14T19:38:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/3109839283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e040ff4b0 0x7f0e0410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/3109839283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e04104b10 0x7f0e04104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/3109839283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040feb00 0x7f0e040fef70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/3109839283 >> 172.21.15.82:0/3109839283 conn(0x7f0e040fa5c0 msgr2=0x7f0e040fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:54.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/3109839283 shutdown_connections 2023-07-14T19:38:54.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/3109839283 wait complete. 2023-07-14T19:38:54.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 Processor -- start 2023-07-14T19:38:54.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- start start 2023-07-14T19:38:54.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040feb00 0x7f0e0419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.617 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 0x7f0e0419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e04104b10 0x7f0e04195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041961b0 con 0x7f0e040ff4b0 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04196320 con 0x7f0e04104b10 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.595+0000 7f0e0c4d2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e041964e0 con 0x7f0e040feb00 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 0x7f0e0419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 0x7f0e0419c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45024/0 (socket says 172.21.15.82:45024) 2023-07-14T19:38:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 -- 172.21.15.82:0/2350050574 learned_addr learned my addr 172.21.15.82:0/2350050574 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e0a26e700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040feb00 0x7f0e0419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e0aa6f700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e04104b10 0x7f0e04195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 -- 172.21.15.82:0/2350050574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040feb00 msgr2=0x7f0e0419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040feb00 0x7f0e0419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 -- 172.21.15.82:0/2350050574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e04104b10 msgr2=0x7f0e04195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e04104b10 0x7f0e04195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 -- 172.21.15.82:0/2350050574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40095f0 con 0x7f0e040ff4b0 2023-07-14T19:38:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e09a6d700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 0x7f0e0419c010 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0df4000c00 tx=0x7f0df40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df401d070 con 0x7f0e040ff4b0 2023-07-14T19:38:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0df4021410 con 0x7f0e040ff4b0 2023-07-14T19:38:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e04196760 con 0x7f0e040ff4b0 2023-07-14T19:38:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4017650 con 0x7f0e040ff4b0 2023-07-14T19:38:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.596+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e041a2ac0 con 0x7f0e040ff4b0 2023-07-14T19:38:54.621 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.597+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e04067430 con 0x7f0e040ff4b0 2023-07-14T19:38:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.598+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0df40217f0 con 0x7f0e040ff4b0 2023-07-14T19:38:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.599+0000 7f0dfb7fe700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0df00a1300 0x7f0df00a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.599+0000 7f0e0a26e700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0df00a1300 0x7f0df00a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.599+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f0df40ce760 con 0x7f0e040ff4b0 2023-07-14T19:38:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.599+0000 7f0e0a26e700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0df00a1300 0x7f0df00a37b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e00005950 tx=0x7f0e0000a300 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.602+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0df4097c00 con 0x7f0e040ff4b0 2023-07-14T19:38:54.810 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.791+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e04196fe0 con 0x7f0e040ff4b0 2023-07-14T19:38:54.811 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.792+0000 7f0dfb7fe700 1 -- 172.21.15.82:0/2350050574 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+22833 (secure 0 0 0) 0x7f0df40b9e60 con 0x7f0e040ff4b0 2023-07-14T19:38:54.812 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:38:54.812 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:38:33.007820+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6801/504215117":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/3311699035":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/1722617918":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/3623265050":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1617843074":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/504215117":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0df00a1300 msgr2=0x7f0df00a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0df00a1300 0x7f0df00a37b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e00005950 tx=0x7f0e0000a300 comp rx=0 tx=0).stop 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 msgr2=0x7f0e0419c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 0x7f0e0419c010 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0df4000c00 tx=0x7f0df40047b0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 shutdown_connections 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0df00a1300 0x7f0df00a37b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.814 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.794+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e04104b10 0x7f0e04195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.795+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e040feb00 0x7f0e0419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.795+0000 7f0e0c4d2700 1 --2- 172.21.15.82:0/2350050574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e040ff4b0 0x7f0e0419c010 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.795+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 >> 172.21.15.82:0/2350050574 conn(0x7f0e040fa5c0 msgr2=0x7f0e040677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:54.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.795+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 shutdown_connections 2023-07-14T19:38:54.815 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.795+0000 7f0e0c4d2700 1 -- 172.21.15.82:0/2350050574 wait complete. 2023-07-14T19:38:54.826 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:38:54.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:38:54.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 -- 172.21.15.82:0/2428233304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd0100e10 msgr2=0x7fccd01011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/2428233304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd0100e10 0x7fccd01011e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009990 tx=0x7fccc0009ca0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 -- 172.21.15.82:0/2428233304 shutdown_connections 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/2428233304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd010a150 0x7fccd010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/2428233304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd0100e10 0x7fccd01011e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/2428233304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd0101720 0x7fccd0109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 -- 172.21.15.82:0/2428233304 >> 172.21.15.82:0/2428233304 conn(0x7fccd00fa5e0 msgr2=0x7fccd00fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 -- 172.21.15.82:0/2428233304 shutdown_connections 2023-07-14T19:38:54.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.932+0000 7fccd4b3b700 1 -- 172.21.15.82:0/2428233304 wait complete. 2023-07-14T19:38:54.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 Processor -- start 2023-07-14T19:38:54.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 -- start start 2023-07-14T19:38:54.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd0100e10 0x7fccd0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd0101720 0x7fccd0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 0x7fccd019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd019ad10 con 0x7fccd010a150 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd019ae80 con 0x7fccd0101720 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccd4b3b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd019b040 con 0x7fccd0100e10 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 0x7fccd019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 0x7fccd019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45040/0 (socket says 172.21.15.82:45040) 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 -- 172.21.15.82:0/371075545 learned_addr learned my addr 172.21.15.82:0/371075545 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fcccdd9b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd0101720 0x7fccd0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccce59c700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd0100e10 0x7fccd0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 -- 172.21.15.82:0/371075545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd0100e10 msgr2=0x7fccd0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd0100e10 0x7fccd0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 -- 172.21.15.82:0/371075545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd0101720 msgr2=0x7fccd0196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd0101720 0x7fccd0196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:54.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.933+0000 7fccced9d700 1 -- 172.21.15.82:0/371075545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc00095f0 con 0x7fccd010a150 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.934+0000 7fccced9d700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 0x7fccd019a6d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fccc800ef30 tx=0x7fccc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.934+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8018070 con 0x7fccd010a150 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.934+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc800f040 con 0x7fccd010a150 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.934+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800cd00 con 0x7fccd010a150 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.934+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd019b2c0 con 0x7fccd010a150 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.934+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd01a2b60 con 0x7fccd010a150 2023-07-14T19:38:54.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.935+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd0105360 con 0x7fccd010a150 2023-07-14T19:38:54.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.936+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fccc8003a00 con 0x7fccd010a150 2023-07-14T19:38:54.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.936+0000 7fccbf7fe700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fccb80a12c0 0x7fccb80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:54.958 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.937+0000 7fccce59c700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fccb80a12c0 0x7fccb80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:54.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.937+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fccc80cd090 con 0x7fccd010a150 2023-07-14T19:38:54.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.937+0000 7fccce59c700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fccb80a12c0 0x7fccb80a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b2e0 tx=0x7fccc0005fd0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:54.959 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:54.940+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fccc8096630 con 0x7fccd010a150 2023-07-14T19:38:55.146 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.127+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccd0104d40 con 0x7fccd010a150 2023-07-14T19:38:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.127+0000 7fccbf7fe700 1 -- 172.21.15.82:0/371075545 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fccc8099c50 con 0x7fccd010a150 2023-07-14T19:38:55.147 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fccb80a12c0 msgr2=0x7fccb80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fccb80a12c0 0x7fccb80a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b2e0 tx=0x7fccc0005fd0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 msgr2=0x7fccd019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 0x7fccd019a6d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fccc800ef30 tx=0x7fccc800c3a0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 shutdown_connections 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fccb80a12c0 0x7fccb80a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccd010a150 0x7fccd019a6d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccd0100e10 0x7fccd0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 --2- 172.21.15.82:0/371075545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd0101720 0x7fccd0196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 >> 172.21.15.82:0/371075545 conn(0x7fccd00fa5e0 msgr2=0x7fccd010c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.129+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 shutdown_connections 2023-07-14T19:38:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.130+0000 7fccd4b3b700 1 -- 172.21.15.82:0/371075545 wait complete. 2023-07-14T19:38:55.161 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:38:55.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 -- 172.21.15.82:0/2799194948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc104b10 msgr2=0x7f14dc104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 --2- 172.21.15.82:0/2799194948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc104b10 0x7f14dc104ee0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009990 tx=0x7f14cc009ca0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 -- 172.21.15.82:0/2799194948 shutdown_connections 2023-07-14T19:38:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 --2- 172.21.15.82:0/2799194948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.294 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 --2- 172.21.15.82:0/2799194948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc104b10 0x7f14dc104ee0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 --2- 172.21.15.82:0/2799194948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc0feb00 0x7f14dc0fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 -- 172.21.15.82:0/2799194948 >> 172.21.15.82:0/2799194948 conn(0x7f14dc0fa5e0 msgr2=0x7f14dc0fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 -- 172.21.15.82:0/2799194948 shutdown_connections 2023-07-14T19:38:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.270+0000 7f14e11b2700 1 -- 172.21.15.82:0/2799194948 wait complete. 2023-07-14T19:38:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.271+0000 7f14e11b2700 1 Processor -- start 2023-07-14T19:38:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.271+0000 7f14e11b2700 1 -- start start 2023-07-14T19:38:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.271+0000 7f14e11b2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc0feb00 0x7f14dc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc104b10 0x7f14dc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc19ad40 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc19aeb0 con 0x7f14dc0feb00 2023-07-14T19:38:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc19b070 con 0x7f14dc104b10 2023-07-14T19:38:55.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc1962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45052/0 (socket says 172.21.15.82:45052) 2023-07-14T19:38:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 -- 172.21.15.82:0/651618084 learned_addr learned my addr 172.21.15.82:0/651618084 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14dad9d700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc0feb00 0x7f14dc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14db59e700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc104b10 0x7f14dc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 -- 172.21.15.82:0/651618084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc104b10 msgr2=0x7f14dc19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc104b10 0x7f14dc19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 -- 172.21.15.82:0/651618084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc0feb00 msgr2=0x7f14dc195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc0feb00 0x7f14dc195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 -- 172.21.15.82:0/651618084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14cc0095f0 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14db59e700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc104b10 0x7f14dc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-14T19:38:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14da59c700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc1962f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d950 tx=0x7f14d000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:55.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0009b40 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc19b350 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14d0009ca0 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d000f620 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.272+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc1a2b90 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.273+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc0673c0 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.275+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f14d0010460 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.275+0000 7f14c3fff700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f14c40a12c0 0x7f14c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.276+0000 7f14dad9d700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f14c40a12c0 0x7f14c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.276+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f14d00cd040 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.276+0000 7f14dad9d700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f14c40a12c0 0x7f14c40a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006010 tx=0x7f14cc00b540 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.279+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f14d00964b0 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.471+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14dc1a2e20 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.472+0000 7f14c3fff700 1 -- 172.21.15.82:0/651618084 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14d0099ad0 con 0x7f14dc0ff4b0 2023-07-14T19:38:55.491 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:38:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f14c40a12c0 msgr2=0x7f14c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f14c40a12c0 0x7f14c40a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006010 tx=0x7f14cc00b540 comp rx=0 tx=0).stop 2023-07-14T19:38:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 msgr2=0x7f14dc1962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc1962f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d950 tx=0x7f14d000dd10 comp rx=0 tx=0).stop 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 shutdown_connections 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f14c40a12c0 0x7f14c40a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14dc104b10 0x7f14dc19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14dc0feb00 0x7f14dc195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 --2- 172.21.15.82:0/651618084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14dc0ff4b0 0x7f14dc1962f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 >> 172.21.15.82:0/651618084 conn(0x7f14dc0fa5e0 msgr2=0x7f14dc102810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 shutdown_connections 2023-07-14T19:38:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.474+0000 7f14e11b2700 1 -- 172.21.15.82:0/651618084 wait complete. 2023-07-14T19:38:55.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:38:55.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 -- 172.21.15.82:0/1366239669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800ff4e0 msgr2=0x7f2e8010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 --2- 172.21.15.82:0/1366239669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800ff4e0 0x7f2e8010b920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800b0a0 tx=0x7f2e7800b3b0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.631 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 -- 172.21.15.82:0/1366239669 shutdown_connections 2023-07-14T19:38:55.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 --2- 172.21.15.82:0/1366239669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800ff4e0 0x7f2e8010b920 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 --2- 172.21.15.82:0/1366239669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 0x7f2e80104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 --2- 172.21.15.82:0/1366239669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e800feb30 0x7f2e800fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 -- 172.21.15.82:0/1366239669 >> 172.21.15.82:0/1366239669 conn(0x7f2e800fa610 msgr2=0x7f2e800fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:55.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 -- 172.21.15.82:0/1366239669 shutdown_connections 2023-07-14T19:38:55.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.612+0000 7f2e848d8700 1 -- 172.21.15.82:0/1366239669 wait complete. 2023-07-14T19:38:55.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 Processor -- start 2023-07-14T19:38:55.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 -- start start 2023-07-14T19:38:55.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 0x7f2e80196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e800ff4e0 0x7f2e801971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 0x7f2e801a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80197b20 con 0x7f2e800feb30 2023-07-14T19:38:55.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801a2260 con 0x7f2e80104b60 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e848d8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e801a2420 con 0x7f2e800ff4e0 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 0x7f2e80196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 0x7f2e80196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45064/0 (socket says 172.21.15.82:45064) 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 -- 172.21.15.82:0/646696142 learned_addr learned my addr 172.21.15.82:0/646696142 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7ed9d700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 0x7f2e801a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7dd9b700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e800ff4e0 0x7f2e801971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 -- 172.21.15.82:0/646696142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e800ff4e0 msgr2=0x7f2e801971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e800ff4e0 0x7f2e801971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 -- 172.21.15.82:0/646696142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 msgr2=0x7f2e801a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 0x7f2e801a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7e59c700 1 -- 172.21.15.82:0/646696142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e78009d00 con 0x7f2e800feb30 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.613+0000 7f2e7ed9d700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 0x7f2e801a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.614+0000 7f2e7e59c700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 0x7f2e80196ca0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000d950 tx=0x7f2e7000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.614+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70009b40 con 0x7f2e800feb30 2023-07-14T19:38:55.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.614+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e70009ca0 con 0x7f2e800feb30 2023-07-14T19:38:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.614+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e801a2700 con 0x7f2e800feb30 2023-07-14T19:38:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.614+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7000f620 con 0x7f2e800feb30 2023-07-14T19:38:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.614+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80194d20 con 0x7f2e800feb30 2023-07-14T19:38:55.638 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.615+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e80066b30 con 0x7f2e800feb30 2023-07-14T19:38:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.616+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2e70010460 con 0x7f2e800feb30 2023-07-14T19:38:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.616+0000 7f2e6f7fe700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2e680a12c0 0x7f2e680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.617+0000 7f2e7dd9b700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2e680a12c0 0x7f2e680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.617+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7f2e700ccfa0 con 0x7f2e800feb30 2023-07-14T19:38:55.639 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.617+0000 7f2e7dd9b700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2e680a12c0 0x7f2e680a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005950 tx=0x7f2e740058e0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:55.640 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.620+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2e70096410 con 0x7f2e800feb30 2023-07-14T19:38:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.809+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e80195160 con 0x7f2e800feb30 2023-07-14T19:38:55.829 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.809+0000 7f2e6f7fe700 1 -- 172.21.15.82:0/646696142 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e70016020 con 0x7f2e800feb30 2023-07-14T19:38:55.829 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:38:55.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2e680a12c0 msgr2=0x7f2e680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2e680a12c0 0x7f2e680a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2e74005950 tx=0x7f2e740058e0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 msgr2=0x7f2e80196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 0x7f2e80196ca0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000d950 tx=0x7f2e7000dd10 comp rx=0 tx=0).stop 2023-07-14T19:38:55.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 shutdown_connections 2023-07-14T19:38:55.831 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2e680a12c0 0x7f2e680a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e80104b60 0x7f2e801a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e800feb30 0x7f2e80196ca0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 --2- 172.21.15.82:0/646696142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e800ff4e0 0x7f2e801971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 >> 172.21.15.82:0/646696142 conn(0x7f2e800fa610 msgr2=0x7f2e800fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:55.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 shutdown_connections 2023-07-14T19:38:55.832 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.811+0000 7f2e848d8700 1 -- 172.21.15.82:0/646696142 wait complete. 2023-07-14T19:38:55.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:55 smithi082 ceph-mon[127461]: pgmap v324: 97 pgs: 97 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2023-07-14T19:38:55.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:55 smithi082 ceph-mon[127461]: from='client.16335 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:38:55.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:55 smithi082 ceph-mon[127461]: from='client.16341 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:38:55.842 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/2350050574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:38:55.843 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/371075545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:38:55.843 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:55 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/651618084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:38:55.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-14T19:38:55.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:55 smithi133 ceph-mon[128894]: pgmap v324: 97 pgs: 97 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2023-07-14T19:38:55.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:55 smithi133 ceph-mon[128894]: from='client.16335 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:38:55.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:55 smithi133 ceph-mon[128894]: from='client.16341 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:38:55.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/2350050574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:38:55.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/371075545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:38:55.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:55 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/651618084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:38:55.967 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/886672273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0100830 msgr2=0x7fa2d01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/886672273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0100830 0x7fa2d01047f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0099c0 tx=0x7fa2cc009cd0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/886672273 shutdown_connections 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/886672273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2d0104ed0 0x7fa2d010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/886672273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2d00ffe90 0x7fa2d0100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/886672273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0100830 0x7fa2d01047f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/886672273 >> 172.21.15.82:0/886672273 conn(0x7fa2d00fb760 msgr2=0x7fa2d00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:55.968 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/886672273 shutdown_connections 2023-07-14T19:38:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.946+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/886672273 wait complete. 2023-07-14T19:38:55.969 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 Processor -- start 2023-07-14T19:38:55.970 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 -- start start 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2d00ffe90 0x7fa2d0195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2d0100830 0x7fa2d01962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 0x7fa2d019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019acf0 con 0x7fa2d0104ed0 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019ae60 con 0x7fa2d0100830 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d7e6b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d019b020 con 0x7fa2d00ffe90 2023-07-14T19:38:55.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d6408700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 0x7fa2d019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d6408700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 0x7fa2d019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45080/0 (socket says 172.21.15.82:45080) 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.947+0000 7fa2d6408700 1 -- 172.21.15.82:0/1596741830 learned_addr learned my addr 172.21.15.82:0/1596741830 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d5406700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2d0100830 0x7fa2d01962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d5c07700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2d00ffe90 0x7fa2d0195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d6408700 1 -- 172.21.15.82:0/1596741830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2d00ffe90 msgr2=0x7fa2d0195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d6408700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2d00ffe90 0x7fa2d0195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.972 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d6408700 1 -- 172.21.15.82:0/1596741830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2d0100830 msgr2=0x7fa2d01962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d6408700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2d0100830 0x7fa2d01962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d6408700 1 -- 172.21.15.82:0/1596741830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0095f0 con 0x7fa2d0104ed0 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d6408700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 0x7fa2d019a6b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8007ee0 tx=0x7fa2c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8017070 con 0x7fa2d0104ed0 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2c800f040 con 0x7fa2d0104ed0 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d019b300 con 0x7fa2d0104ed0 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005740 con 0x7fa2d0104ed0 2023-07-14T19:38:55.973 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.948+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d01a2b40 con 0x7fa2d0104ed0 2023-07-14T19:38:55.974 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.949+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d0067430 con 0x7fa2d0104ed0 2023-07-14T19:38:55.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.950+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa2c80058a0 con 0x7fa2d0104ed0 2023-07-14T19:38:55.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.951+0000 7fa2c6ffd700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa2bc0a1300 0x7fa2bc0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:38:55.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.951+0000 7fa2d5c07700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa2bc0a1300 0x7fa2bc0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:38:55.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.951+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8426+0+0 (secure 0 0 0) 0x7fa2c80ccb60 con 0x7fa2d0104ed0 2023-07-14T19:38:55.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.951+0000 7fa2d5c07700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa2bc0a1300 0x7fa2bc0a37b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005950 tx=0x7fa2c00058e0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:38:55.975 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:55.954+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fa2c8095fd0 con 0x7fa2d0104ed0 2023-07-14T19:38:56.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:55 smithi161 ceph-mon[129184]: pgmap v324: 97 pgs: 97 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 369 op/s 2023-07-14T19:38:56.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:55 smithi161 ceph-mon[129184]: from='client.16335 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:38:56.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:55 smithi161 ceph-mon[129184]: from='client.16341 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-14T19:38:56.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:55 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/2350050574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:38:56.011 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:55 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/371075545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:38:56.011 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:55 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/651618084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:38:56.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.195+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fa2d01a2d70 con 0x7fa2d0104ed0 2023-07-14T19:38:56.222 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.203+0000 7fa2c6ffd700 1 -- 172.21.15.82:0/1596741830 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+733775 (secure 0 0 0) 0x7fa2c80995f0 con 0x7fa2d0104ed0 2023-07-14T19:38:56.229 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:38:56.229 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:38:56.229 INFO:teuthology.orchestra.run.smithi082.stdout: "epoch": 24, 2023-07-14T19:38:56.229 INFO:teuthology.orchestra.run.smithi082.stdout: "active_gid": 15414, 2023-07-14T19:38:56.229 INFO:teuthology.orchestra.run.smithi082.stdout: "active_name": "x", 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addrs": { 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6800", 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2084410629 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.230 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:6801", 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2084410629 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.231 INFO:teuthology.orchestra.run.smithi082.stdout: "active_addr": "172.21.15.82:6801/2084410629", 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "active_change": "2023-07-14T19:28:33.481736+0000", 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "active_mgr_features": 4540138322906710015, 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "available": true, 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "standbys": [ 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 15423, 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "y", 2023-07-14T19:38:56.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540138322906710015, 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.233 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.234 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.235 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.236 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.236 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.236 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.236 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.236 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.237 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.240 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.240 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.240 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.240 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.240 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.240 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.242 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.244 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.246 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.247 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2023-07-14T19:38:56.248 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2023-07-14T19:38:56.249 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.250 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.251 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.252 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.254 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.255 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.256 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2023-07-14T19:38:56.257 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.258 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.259 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.260 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2023-07-14T19:38:56.262 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.263 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2023-07-14T19:38:56.264 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.265 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2023-07-14T19:38:56.266 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2023-07-14T19:38:56.267 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T19:38:56.268 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.269 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T19:38:56.270 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.271 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2023-07-14T19:38:56.273 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.274 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.275 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.276 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.277 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.278 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.279 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.280 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.281 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.282 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.283 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.284 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2023-07-14T19:38:56.285 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2023-07-14T19:38:56.286 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap" 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.287 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.288 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:56.289 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T19:38:56.292 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.293 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.294 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.295 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.296 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2023-07-14T19:38:56.297 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.298 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.299 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.300 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: "alertmanager_web_password": { 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alertmanager_web_password", 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.301 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Alertmanager web password", 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.302 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "alertmanager_web_user": { 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alertmanager_web_user", 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.303 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Alertmanager web user", 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.304 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.305 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2023-07-14T19:38:56.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.307 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.308 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.309 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.310 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.311 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.312 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2023-07-14T19:38:56.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.314 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.315 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.316 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.317 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.318 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T19:38:56.319 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.320 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.321 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2023-07-14T19:38:56.322 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.323 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.324 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.326 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.327 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T19:38:56.328 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.329 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.330 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2023-07-14T19:38:56.331 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2023-07-14T19:38:56.332 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.333 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T19:38:56.335 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.336 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.337 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2023-07-14T19:38:56.338 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.339 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2023-07-14T19:38:56.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.341 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.341 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.341 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.341 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.341 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.341 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2023-07-14T19:38:56.342 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.343 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T19:38:56.344 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.344 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.344 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.344 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.344 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2023-07-14T19:38:56.344 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.345 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.346 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.347 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.348 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.349 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.352 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2023-07-14T19:38:56.353 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.354 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.355 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.356 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.357 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.358 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.359 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.360 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.361 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.362 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.363 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.364 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.365 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.366 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.367 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2023-07-14T19:38:56.368 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.369 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.370 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2023-07-14T19:38:56.371 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T19:38:56.372 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.373 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.374 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2023-07-14T19:38:56.375 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T19:38:56.376 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.377 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.378 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_web_password": { 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_web_password", 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.379 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus web password", 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_web_user": { 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_web_user", 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.381 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus web user", 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2023-07-14T19:38:56.382 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T19:38:56.383 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.384 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.385 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.386 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.389 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.390 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2023-07-14T19:38:56.391 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2023-07-14T19:38:56.392 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.393 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.394 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.395 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.398 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2023-07-14T19:38:56.400 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T19:38:56.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.403 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.404 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.405 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.407 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.408 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.409 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.410 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.411 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.412 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.414 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.415 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.416 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.417 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.418 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.419 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.420 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2023-07-14T19:38:56.423 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.424 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T19:38:56.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.426 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.427 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.428 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.429 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.430 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.431 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T19:38:56.432 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.433 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.434 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.435 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.436 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.437 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.438 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.440 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T19:38:56.441 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.442 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T19:38:56.443 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.444 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.445 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.446 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.447 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.449 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T19:38:56.450 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.451 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.452 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.453 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.454 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.456 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T19:38:56.457 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.458 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.459 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.459 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.459 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.459 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.459 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.460 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T19:38:56.461 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.462 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T19:38:56.463 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.464 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.465 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.466 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.467 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.468 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.469 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T19:38:56.470 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.471 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.472 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.473 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.474 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.475 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.476 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T19:38:56.477 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.478 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.479 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.480 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.482 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.483 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.485 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T19:38:56.486 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.487 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.489 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.492 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.493 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.494 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2023-07-14T19:38:56.495 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.499 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.501 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2023-07-14T19:38:56.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.504 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.505 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.510 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.511 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.512 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.513 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:56.518 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.519 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2023-07-14T19:38:56.520 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.521 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.522 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.523 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2023-07-14T19:38:56.524 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.525 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.526 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.528 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.529 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2023-07-14T19:38:56.530 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.531 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.532 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.533 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.534 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.535 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.536 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.538 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.539 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.541 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.543 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2023-07-14T19:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.547 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.548 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.549 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.550 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.552 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.553 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.554 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.555 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.556 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.557 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.558 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.559 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.560 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.561 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.562 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2023-07-14T19:38:56.563 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.564 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.565 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.566 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:56.567 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.568 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2023-07-14T19:38:56.569 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.570 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.571 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.572 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.573 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2023-07-14T19:38:56.575 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.576 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.577 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.578 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.579 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.580 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.581 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.582 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.583 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.584 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.585 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.586 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2023-07-14T19:38:56.587 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2023-07-14T19:38:56.588 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.589 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.590 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:56.591 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.592 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.593 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2023-07-14T19:38:56.594 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T19:38:56.595 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.596 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.597 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.598 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.599 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.600 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.601 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.602 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.603 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.604 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.605 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.606 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.607 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.608 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.609 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.610 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.611 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.612 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.613 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.614 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.615 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.616 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2023-07-14T19:38:56.618 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T19:38:56.619 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.620 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.621 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.623 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.624 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.625 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.626 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.627 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.628 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2023-07-14T19:38:56.629 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.630 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.632 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.633 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.634 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.635 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.636 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.637 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2023-07-14T19:38:56.640 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.641 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.642 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.643 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.644 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.645 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.646 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.647 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.648 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.649 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.650 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.651 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.652 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.653 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.654 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.656 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.657 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.659 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.660 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.661 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.663 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.664 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.665 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.666 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.669 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.671 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.674 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.675 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.676 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.677 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.678 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.679 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.680 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.681 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.682 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.683 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.684 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.685 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.688 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2023-07-14T19:38:56.691 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.693 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.694 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.695 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.696 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.699 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.700 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.701 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.702 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.703 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.704 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.705 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.706 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.707 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.708 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.709 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.710 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.711 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.712 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.713 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.714 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.715 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.716 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.717 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.718 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "noautoscale": { 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "noautoscale", 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "global autoscale flag", 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:56.723 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2023-07-14T19:38:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2023-07-14T19:38:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2023-07-14T19:38:56.727 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.728 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.729 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.731 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.733 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.734 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.735 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.737 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.738 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.739 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2023-07-14T19:38:56.740 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.741 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.742 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.743 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.744 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.745 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.746 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.747 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T19:38:56.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T19:38:56.748 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.748 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.748 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.749 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.750 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.751 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.752 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.753 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.754 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.755 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.756 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2023-07-14T19:38:56.757 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.759 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.760 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2023-07-14T19:38:56.761 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.762 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.763 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:56.764 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.765 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.766 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.767 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.768 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.769 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.770 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.771 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.772 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.774 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.775 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.776 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.777 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.778 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.779 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.780 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2023-07-14T19:38:56.783 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.785 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.786 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.787 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.788 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.789 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.790 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.791 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.792 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.793 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.794 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.795 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.796 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.796 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.796 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.796 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.796 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.797 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.798 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.799 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.800 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.801 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:56.802 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "drive_group_interval": { 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "drive_group_interval", 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300.0", 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2023-07-14T19:38:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.830 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.831 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.832 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.833 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.834 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.835 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.836 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.837 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.838 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2023-07-14T19:38:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.841 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2023-07-14T19:38:56.842 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.843 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.844 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.845 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.846 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.847 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.848 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2023-07-14T19:38:56.849 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.850 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.851 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.852 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.853 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.854 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.855 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2023-07-14T19:38:56.856 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.857 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.858 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.859 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.860 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.861 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.862 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.863 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.864 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.865 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.866 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.867 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.868 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.869 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.870 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.871 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.873 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.874 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.875 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.876 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.877 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.879 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.880 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.881 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.882 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.883 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.884 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.885 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.886 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.887 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.889 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.890 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.891 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T19:38:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.893 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.894 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.895 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.896 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.897 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.898 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.900 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.901 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.902 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.904 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.905 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.906 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.907 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.908 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.909 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2023-07-14T19:38:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.911 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.912 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.914 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.915 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.916 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.917 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.918 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2023-07-14T19:38:56.919 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.920 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.921 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.922 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.923 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.924 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.925 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2023-07-14T19:38:56.926 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.927 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.928 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.929 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.930 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.931 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.933 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.934 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.935 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.936 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.937 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.938 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.939 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.940 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.941 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.942 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.943 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.944 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.945 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.946 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.947 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.949 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.950 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.951 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.952 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.953 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.954 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.955 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.956 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.958 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.959 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.960 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.960 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.960 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.960 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.961 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2023-07-14T19:38:56.962 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T19:38:56.963 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.964 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.965 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.966 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2023-07-14T19:38:56.967 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.968 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.969 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:56.970 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.971 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.972 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.973 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.974 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.975 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.976 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.977 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.978 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.979 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.980 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.981 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.982 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2023-07-14T19:38:56.983 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.984 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.985 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:56.986 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: "gid": 25048, 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "z", 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr_features": 4540138322906710015, 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:56.987 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.988 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.989 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.990 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.991 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.992 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:56.994 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:56.995 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.996 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.997 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:56.998 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.000 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.001 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2023-07-14T19:38:57.002 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.003 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.005 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.006 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.007 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.008 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2023-07-14T19:38:57.009 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.010 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.011 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.012 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.013 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.014 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.015 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.016 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2023-07-14T19:38:57.017 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.018 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.019 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T19:38:57.020 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.021 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.022 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2023-07-14T19:38:57.023 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.024 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.025 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2023-07-14T19:38:57.026 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.027 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.028 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.029 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.030 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.031 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.032 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.033 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.034 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.035 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.036 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.037 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.038 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.039 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap" 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2023-07-14T19:38:57.040 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.041 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.042 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.043 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.044 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:57.045 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2023-07-14T19:38:57.046 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.047 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.048 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.049 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2023-07-14T19:38:57.051 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.052 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.053 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "alertmanager_web_password": { 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alertmanager_web_password", 2023-07-14T19:38:57.054 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Alertmanager web password", 2023-07-14T19:38:57.055 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: "alertmanager_web_user": { 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alertmanager_web_user", 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.056 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Alertmanager web user", 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.057 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.058 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.059 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.060 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2023-07-14T19:38:57.061 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.062 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2023-07-14T19:38:57.063 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T19:38:57.064 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.065 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.066 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.067 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.068 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.069 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2023-07-14T19:38:57.070 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.071 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2023-07-14T19:38:57.072 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2023-07-14T19:38:57.073 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T19:38:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2023-07-14T19:38:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2023-07-14T19:38:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2023-07-14T19:38:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T19:38:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2023-07-14T19:38:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T19:38:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2023-07-14T19:38:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2023-07-14T19:38:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2023-07-14T19:38:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T19:38:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T19:38:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2023-07-14T19:38:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2023-07-14T19:38:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2023-07-14T19:38:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T19:38:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_web_password": { 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_web_password", 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus web password", 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_web_user": { 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_web_user", 2023-07-14T19:38:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus web user", 2023-07-14T19:38:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2023-07-14T19:38:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T19:38:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2023-07-14T19:38:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2023-07-14T19:38:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.156 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.157 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.158 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.159 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.160 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.161 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.162 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.163 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.164 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.165 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.166 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.167 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2023-07-14T19:38:57.168 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.169 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T19:38:57.170 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.171 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T19:38:57.172 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.177 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T19:38:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.180 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.181 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.182 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.183 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.184 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.185 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T19:38:57.186 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.188 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.189 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.191 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.192 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.193 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.194 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T19:38:57.195 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.196 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.197 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.198 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.199 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.200 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.201 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T19:38:57.202 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.203 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T19:38:57.204 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.205 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.206 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.207 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.208 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.209 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.210 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T19:38:57.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.212 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.213 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.214 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.215 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.216 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.217 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T19:38:57.218 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.219 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.220 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.221 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.222 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.224 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T19:38:57.225 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.226 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.227 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.228 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.229 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.231 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T19:38:57.232 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.233 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T19:38:57.234 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.235 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.236 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.237 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.238 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.239 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T19:38:57.241 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.242 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.243 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.244 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.246 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.247 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.248 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.249 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2023-07-14T19:38:57.250 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2023-07-14T19:38:57.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.252 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.254 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.255 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.256 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.257 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.258 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.261 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2023-07-14T19:38:57.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.267 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.268 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2023-07-14T19:38:57.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.271 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.272 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2023-07-14T19:38:57.273 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.275 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2023-07-14T19:38:57.276 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2023-07-14T19:38:57.277 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.278 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.279 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.280 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.281 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.282 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.283 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.284 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.286 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.287 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.288 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.289 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.290 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.291 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2023-07-14T19:38:57.292 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.293 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.294 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.295 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.296 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2023-07-14T19:38:57.298 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.299 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:57.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T19:38:57.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.303 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.304 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.305 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.306 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.307 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.310 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.311 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.312 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.313 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.315 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2023-07-14T19:38:57.316 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.317 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.319 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2023-07-14T19:38:57.320 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.321 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.322 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.324 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.324 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T19:38:57.324 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.324 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.324 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.324 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.325 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2023-07-14T19:38:57.325 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2023-07-14T19:38:57.325 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.325 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.325 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.325 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.326 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2023-07-14T19:38:57.327 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.328 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.330 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.331 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.332 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.333 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.334 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.335 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.336 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2023-07-14T19:38:57.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2023-07-14T19:38:57.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.340 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.341 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.343 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.344 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.345 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.346 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.347 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.349 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.350 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.351 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.352 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.353 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.354 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.355 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.356 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.357 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.358 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.360 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.361 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.362 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.363 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.364 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.365 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.366 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.367 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2023-07-14T19:38:57.368 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.369 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2023-07-14T19:38:57.371 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.372 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.373 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.374 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.375 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.376 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.377 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.378 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.379 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.380 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.381 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2023-07-14T19:38:57.382 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2023-07-14T19:38:57.383 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.384 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.385 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.386 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.387 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.388 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.389 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.390 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.391 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.392 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.393 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.394 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2023-07-14T19:38:57.395 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.396 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.397 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2023-07-14T19:38:57.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2023-07-14T19:38:57.399 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.400 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.401 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.402 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.403 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.404 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.405 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.406 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.407 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.408 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.409 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.410 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.411 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.411 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.411 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.411 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.411 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.411 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2023-07-14T19:38:57.412 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.413 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.414 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.415 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.416 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.417 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.418 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.419 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.420 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.421 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.422 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.423 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.424 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.426 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.427 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.428 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.430 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.431 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2023-07-14T19:38:57.432 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.433 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.434 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.435 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.436 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.437 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.439 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.440 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.441 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.442 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.443 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2023-07-14T19:38:57.444 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.446 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.447 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.448 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.449 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.451 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.452 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.453 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.454 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.455 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2023-07-14T19:38:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.456 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.457 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.458 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.459 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.460 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.461 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.462 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.463 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.464 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.465 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.466 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.467 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.468 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.469 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.470 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.471 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.472 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.473 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.474 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.475 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.476 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "noautoscale": { 2023-07-14T19:38:57.477 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "noautoscale", 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.478 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "global autoscale flag", 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.479 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.480 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.481 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.482 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.483 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.484 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2023-07-14T19:38:57.485 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.486 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.487 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.488 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.489 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.492 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.493 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.494 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.495 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.496 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.497 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.498 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.499 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.500 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.501 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2023-07-14T19:38:57.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.504 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.505 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.507 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.510 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.511 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.512 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.513 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:57.518 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.519 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.520 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.521 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.524 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.526 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.527 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.528 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.529 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.530 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.533 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.534 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.536 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.537 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.538 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T19:38:57.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.540 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.541 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.542 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.543 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.544 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.545 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.546 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.547 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.548 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.549 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.550 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.552 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.553 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.554 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.555 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.556 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.557 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.558 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.559 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.560 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.561 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.562 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.563 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.564 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.565 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.566 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.567 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.568 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: "drive_group_interval": { 2023-07-14T19:38:57.570 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "drive_group_interval", 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300.0", 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.571 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.572 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.573 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.574 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.576 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.577 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.578 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.579 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.580 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.581 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.582 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.583 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.584 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.585 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.586 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.587 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.588 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.589 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.590 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.591 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.592 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.593 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.594 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.595 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2023-07-14T19:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.597 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.598 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.599 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.600 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.601 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.602 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2023-07-14T19:38:57.603 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.604 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.605 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.606 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.607 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.608 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2023-07-14T19:38:57.610 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2023-07-14T19:38:57.612 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.613 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2023-07-14T19:38:57.614 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.615 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.616 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.617 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.618 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.619 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.620 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.622 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.623 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.624 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.625 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.626 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.627 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.628 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.629 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.630 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.633 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.634 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.635 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.636 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2023-07-14T19:38:57.637 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.638 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.639 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.640 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.642 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.643 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.644 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.645 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.646 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.647 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.648 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:57.649 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.650 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.651 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.652 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.653 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.657 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.658 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.659 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.660 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.661 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.662 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.663 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.664 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.665 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.667 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.669 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2023-07-14T19:38:57.670 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.671 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2023-07-14T19:38:57.672 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.673 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.674 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.675 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.676 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.677 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.678 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2023-07-14T19:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.680 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.681 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.682 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.683 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.684 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.685 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.686 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.687 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.688 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.689 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.690 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.691 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.692 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.694 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.695 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.696 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2023-07-14T19:38:57.697 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.698 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.698 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.698 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T19:38:57.698 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.698 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.698 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.699 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.699 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.699 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.699 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.699 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.700 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.701 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.702 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.703 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.705 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.706 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.706 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.706 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.706 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.706 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.706 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.707 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.707 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.707 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.707 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.707 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.707 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.708 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.711 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.713 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.714 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.715 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.716 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.717 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.717 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.717 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.717 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.717 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.718 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.719 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.720 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2023-07-14T19:38:57.721 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.722 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.722 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.722 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2023-07-14T19:38:57.722 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.722 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.723 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.723 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T19:38:57.723 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.723 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.723 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.723 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.724 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.725 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.726 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.726 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.726 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2023-07-14T19:38:57.726 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.726 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.726 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2023-07-14T19:38:57.727 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.728 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.728 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.728 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.728 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.728 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.728 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.729 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.731 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.732 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.733 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.734 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.735 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.736 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.736 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.736 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.736 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.736 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.737 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.738 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.738 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.738 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.738 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.738 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.739 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.739 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.739 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.739 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.740 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.740 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.740 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.740 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.740 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.740 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.741 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.742 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.743 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.744 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.744 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.744 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.744 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2023-07-14T19:38:57.745 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.746 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.747 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.748 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.749 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.749 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.749 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.749 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.749 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.749 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:57.750 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.750 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.750 INFO:teuthology.orchestra.run.smithi082.stdout: "modules": [ 2023-07-14T19:38:57.750 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2023-07-14T19:38:57.750 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard", 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: "iostat", 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: "nfs", 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: "restful", 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: "snap_schedule" 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: "available_modules": [ 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.751 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alerts", 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.752 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How frequently to reexamine health status", 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.753 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.754 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.755 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.756 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.757 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.758 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.759 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.760 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.760 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.760 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.760 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.760 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.761 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.761 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.761 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.761 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.761 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.761 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.762 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.763 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.763 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.763 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.763 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.763 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.764 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_destination": { 2023-07-14T19:38:57.764 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_destination", 2023-07-14T19:38:57.764 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.764 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.764 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email address to send alerts to", 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.765 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_from_name": { 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_from_name", 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.766 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "Ceph", 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Email From: name", 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.767 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_host": { 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_host", 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.768 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP server", 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.769 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.770 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_password": { 2023-07-14T19:38:57.770 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_password", 2023-07-14T19:38:57.770 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.770 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.770 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.770 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.771 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.771 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.771 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.771 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Password to authenticate with", 2023-07-14T19:38:57.771 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_port": { 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_port", 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.772 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "465", 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP port", 2023-07-14T19:38:57.773 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_sender": { 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_sender", 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.774 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.775 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.775 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.775 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.775 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.775 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SMTP envelope sender", 2023-07-14T19:38:57.775 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.776 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.776 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.776 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.776 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_ssl": { 2023-07-14T19:38:57.776 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_ssl", 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.777 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: "smtp_user": { 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "smtp_user", 2023-07-14T19:38:57.778 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "User to authenticate as", 2023-07-14T19:38:57.779 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.780 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "balancer", 2023-07-14T19:38:57.781 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.781 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.781 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.781 INFO:teuthology.orchestra.run.smithi082.stdout: "active": { 2023-07-14T19:38:57.781 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "active", 2023-07-14T19:38:57.781 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically balance PGs across cluster", 2023-07-14T19:38:57.782 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_time": { 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_time", 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.783 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0000", 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "beginning time of day to automatically balance", 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T19:38:57.784 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "begin_weekday": { 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "begin_weekday", 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.785 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.786 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_max_iterations": { 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_max_iterations", 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "25", 2023-07-14T19:38:57.787 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "250", 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.788 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_metrics": { 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_metrics", 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "pgs,objects,bytes", 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.789 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "crush_compat_step": { 2023-07-14T19:38:57.790 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crush_compat_step", 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.5", 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0.001", 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "0.999", 2023-07-14T19:38:57.791 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.792 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "aggressiveness of optimization", 2023-07-14T19:38:57.792 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-14T19:38:57.792 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.792 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.792 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.792 INFO:teuthology.orchestra.run.smithi082.stdout: "end_time": { 2023-07-14T19:38:57.793 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_time", 2023-07-14T19:38:57.793 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.793 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.793 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.793 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2359", 2023-07-14T19:38:57.793 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ending time of day to automatically balance", 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.794 INFO:teuthology.orchestra.run.smithi082.stdout: "end_weekday": { 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "end_weekday", 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "0", 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "6", 2023-07-14T19:38:57.795 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.796 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-14T19:38:57.796 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-14T19:38:57.796 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.796 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.796 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.797 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.797 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.797 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.797 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.797 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.798 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.798 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.798 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.798 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.799 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.799 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.799 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.799 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.800 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.800 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.800 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.801 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.801 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.801 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.802 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.802 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.802 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.803 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.803 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.803 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.804 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.804 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.804 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.804 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.805 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.805 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.805 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.806 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.806 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.806 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.806 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.806 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.807 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.807 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.807 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.807 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.808 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.808 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.808 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.808 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.808 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.809 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.809 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.809 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.809 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.809 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.809 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.810 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.810 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.810 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.810 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.810 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.810 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.811 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.811 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.811 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.811 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.812 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.812 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.812 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.812 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.812 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.812 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.813 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.813 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.813 INFO:teuthology.orchestra.run.smithi082.stdout: "min_score": { 2023-07-14T19:38:57.813 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_score", 2023-07-14T19:38:57.813 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.813 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2023-07-14T19:38:57.815 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.816 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.816 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.816 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "upmap", 2023-07-14T19:38:57.816 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.816 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.816 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.817 INFO:teuthology.orchestra.run.smithi082.stdout: "crush-compat", 2023-07-14T19:38:57.817 INFO:teuthology.orchestra.run.smithi082.stdout: "none", 2023-07-14T19:38:57.817 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap" 2023-07-14T19:38:57.817 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.817 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Balancer mode", 2023-07-14T19:38:57.817 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.818 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.818 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.818 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.818 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_ids": { 2023-07-14T19:38:57.818 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_ids", 2023-07-14T19:38:57.818 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.819 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.819 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.819 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.819 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.819 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.819 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:57.820 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.821 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_deviation": { 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_deviation", 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.822 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.823 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: "upmap_max_optimizations": { 2023-07-14T19:38:57.824 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "upmap_max_optimizations", 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.825 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.826 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cephadm", 2023-07-14T19:38:57.827 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.827 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.827 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.827 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_down_multiplier": { 2023-07-14T19:38:57.827 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_down_multiplier", 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.828 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_refresh_rate": { 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_refresh_rate", 2023-07-14T19:38:57.829 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "20", 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-14T19:38:57.830 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "agent_starting_port": { 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "agent_starting_port", 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.831 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4721", 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.832 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "alertmanager_web_password": { 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alertmanager_web_password", 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Alertmanager web password", 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.834 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "alertmanager_web_user": { 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "alertmanager_web_user", 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.835 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Alertmanager web user", 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.836 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_ptrace": { 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_ptrace", 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.837 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.838 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-14T19:38:57.838 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-14T19:38:57.838 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.838 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.838 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.838 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_interval": { 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_interval", 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.839 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.840 INFO:teuthology.orchestra.run.smithi082.stdout: "autotune_memory_target_ratio": { 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "autotune_memory_target_ratio", 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0.7", 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.841 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: "cgroups_split": { 2023-07-14T19:38:57.842 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cgroups_split", 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.843 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: "config_checks_enabled": { 2023-07-14T19:38:57.844 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_checks_enabled", 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.845 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "config_dashboard": { 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_dashboard", 2023-07-14T19:38:57.846 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.847 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_alertmanager": { 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_alertmanager", 2023-07-14T19:38:57.848 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.849 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_base": { 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_base", 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Container image name, without the tag", 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.851 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_elasticsearch": { 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_elasticsearch", 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.852 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "elasticsearch container image", 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_grafana": { 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_grafana", 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.854 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_haproxy": { 2023-07-14T19:38:57.855 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_haproxy", 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.856 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "HAproxy container image", 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_agent": { 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_agent", 2023-07-14T19:38:57.857 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger agent container image", 2023-07-14T19:38:57.858 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_collector": { 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_collector", 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.859 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger collector container image", 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.860 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_jaeger_query": { 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_jaeger_query", 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.861 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Jaeger query container image", 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.862 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_keepalived": { 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_keepalived", 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-14T19:38:57.863 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Keepalived container image", 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.864 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_loki": { 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_loki", 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.865 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Loki container image", 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_node_exporter": { 2023-07-14T19:38:57.866 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_node_exporter", 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.867 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_prometheus": { 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_prometheus", 2023-07-14T19:38:57.868 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus container image", 2023-07-14T19:38:57.869 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_promtail": { 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_promtail", 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.870 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Promtail container image", 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.871 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: "container_image_snmp_gateway": { 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_image_snmp_gateway", 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.872 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "SNMP Gateway container image", 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.873 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "container_init": { 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "container_init", 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.874 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Run podman/docker with `--init`", 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.875 INFO:teuthology.orchestra.run.smithi082.stdout: "daemon_cache_timeout": { 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "daemon_cache_timeout", 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.876 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "default_cephadm_command_timeout": { 2023-07-14T19:38:57.877 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_cephadm_command_timeout", 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "900", 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.878 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-14T19:38:57.879 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.879 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.879 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.880 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.880 INFO:teuthology.orchestra.run.smithi082.stdout: "default_registry": { 2023-07-14T19:38:57.880 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "default_registry", 2023-07-14T19:38:57.880 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "docker.io", 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-14T19:38:57.881 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "device_cache_timeout": { 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_cache_timeout", 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.882 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1800", 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache device inventory", 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.883 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "device_enhanced_scan": { 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_enhanced_scan", 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.884 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.885 INFO:teuthology.orchestra.run.smithi082.stdout: "facts_cache_timeout": { 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "facts_cache_timeout", 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.886 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "seconds to cache host facts data", 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "host_check_interval": { 2023-07-14T19:38:57.887 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "host_check_interval", 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.888 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to perform a host check", 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.889 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.891 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "log_refresh_metadata": { 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_refresh_metadata", 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.892 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.894 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.895 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.896 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.897 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.898 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.899 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.900 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.901 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "*", 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.902 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "max_count_per_host": { 2023-07-14T19:38:57.903 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_count_per_host", 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.904 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of daemons per service per host", 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "max_osd_draining_count": { 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_osd_draining_count", 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.905 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-14T19:38:57.906 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "migration_current": { 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "migration_current", 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.907 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "internal - do not modify", 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.908 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": { 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mode", 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "root", 2023-07-14T19:38:57.909 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm-package", 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "root" 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "mode for remote execution of cephadm", 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.910 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_alerts_path": { 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_alerts_path", 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.911 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_web_password": { 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_web_password", 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.913 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus web password", 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "prometheus_web_user": { 2023-07-14T19:38:57.914 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus_web_user", 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.915 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Prometheus web user", 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_insecure": { 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_insecure", 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.916 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.917 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_password": { 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_password", 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.918 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.919 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_url": { 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_url", 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.920 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.921 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "registry_username": { 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "registry_username", 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.922 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "secure_monitoring_stack": { 2023-07-14T19:38:57.923 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "secure_monitoring_stack", 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.924 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "service_discovery_port": { 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "service_discovery_port", 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.925 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8765", 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "cephadm service discovery port", 2023-07-14T19:38:57.926 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "ssh_config_file": { 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssh_config_file", 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.927 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.928 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "use_agent": { 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_agent", 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.929 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.930 INFO:teuthology.orchestra.run.smithi082.stdout: "use_repo_digest": { 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "use_repo_digest", 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.931 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_failed_host_check": { 2023-07-14T19:38:57.932 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_failed_host_check", 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.933 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if the host check fails", 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_daemons": { 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_daemons", 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.934 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.935 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_on_stray_hosts": { 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_on_stray_hosts", 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.936 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.937 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.938 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.938 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crash", 2023-07-14T19:38:57.938 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.939 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.939 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.940 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.941 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:57.942 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.943 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.944 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:57.945 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.947 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.948 INFO:teuthology.orchestra.run.smithi082.stdout: "retain_interval": { 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retain_interval", 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "31536000", 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.949 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_recent_interval": { 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_recent_interval", 2023-07-14T19:38:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1209600", 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.951 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:57.952 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dashboard", 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.953 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.954 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_HOST": { 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.955 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.956 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.957 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_ENABLED": { 2023-07-14T19:38:57.958 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_ENABLED", 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.959 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-14T19:38:57.960 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.961 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "ENABLE_BROWSABLE_API": { 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.962 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.963 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.964 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.965 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.966 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-14T19:38:57.967 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.968 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.969 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.970 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.971 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.972 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.973 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-14T19:38:57.974 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.975 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.976 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.977 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_PASSWORD": { 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.978 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.979 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.980 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.981 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_URL": { 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_URL", 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_API_USERNAME": { 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-14T19:38:57.983 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.984 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.985 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.986 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.987 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.988 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.989 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-14T19:38:57.990 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.991 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_HOST": { 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:57.992 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.994 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.995 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.996 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:57.998 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-14T19:38:57.999 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.000 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.001 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.002 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.003 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.004 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.005 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-14T19:38:58.006 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.007 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_ENABLED": { 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.008 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.009 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.010 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.011 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.012 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-14T19:38:58.013 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8", 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.014 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-14T19:38:58.015 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "45", 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.016 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ACCESS_KEY": { 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.017 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.018 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "admin", 2023-07-14T19:38:58.019 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.020 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SECRET_KEY": { 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.021 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "RGW_API_SSL_VERIFY": { 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-14T19:38:58.022 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.023 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.024 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.025 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.027 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.028 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "cross_origin_url": { 2023-07-14T19:38:58.029 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cross_origin_url", 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.030 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "crt_file": { 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "crt_file", 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.031 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "debug": { 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "debug", 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.033 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Enable/disable debug options", 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.034 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "jwt_token_ttl": { 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "jwt_token_ttl", 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "28800", 2023-07-14T19:38:58.035 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.036 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.037 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.038 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.039 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.040 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.041 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.042 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.043 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.044 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.045 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.046 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "motd": { 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "motd", 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.047 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "The message of the day", 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.048 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect_resolve_ip_addr": { 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "redirect_resolve_ip_addr", 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.049 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.050 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.051 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.052 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8080", 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2023-07-14T19:38:58.054 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.055 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl_server_port": { 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl_server_port", 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.056 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8443", 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.057 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "redirect", 2023-07-14T19:38:58.058 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "redirect" 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.059 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.061 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "url_prefix": { 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url_prefix", 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.062 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_monitoring": { 2023-07-14T19:38:58.064 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_monitoring", 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.065 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "monitor device health metrics", 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.066 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.067 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.068 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.069 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.070 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.071 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.072 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.073 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.074 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "mark_out_threshold": { 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mark_out_threshold", 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.075 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "2419200", 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.076 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "pool_name": { 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pool_name", 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "device_health_metrics", 2023-07-14T19:38:58.077 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.078 INFO:teuthology.orchestra.run.smithi082.stdout: "retention_period": { 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "retention_period", 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15552000", 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.079 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long to retain device health metrics", 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_frequency": { 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_frequency", 2023-07-14T19:38:58.080 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-14T19:38:58.081 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "self_heal": { 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "self_heal", 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.082 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.083 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.084 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how frequently to wake up and check device health", 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "warn_threshold": { 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "warn_threshold", 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7257600", 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.086 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "diskprediction_local", 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.088 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.089 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.090 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.091 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.093 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.094 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.095 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.096 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "predict_interval": { 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predict_interval", 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.097 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "86400", 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.098 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "predictor_model": { 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "predictor_model", 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.099 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "prophetstor", 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.100 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "600", 2023-07-14T19:38:58.101 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "influx", 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": false, 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "influxdb python module not found", 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: "batch_size": { 2023-07-14T19:38:58.103 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "batch_size", 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5000", 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.104 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "database": { 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "database", 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.105 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "ceph", 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.106 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": { 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "hostname", 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.107 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server hostname", 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "30", 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "5", 2023-07-14T19:38:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.110 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.111 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.112 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.112 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.112 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.112 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.112 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.112 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.113 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.113 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.113 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.113 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.113 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.114 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.115 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.116 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.117 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.118 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.119 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "password": { 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "password", 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.120 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "password of InfluxDB server user", 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.121 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "port": { 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "port", 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "8086", 2023-07-14T19:38:58.122 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.123 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.123 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.123 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "InfluxDB server port", 2023-07-14T19:38:58.123 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.123 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "ssl": { 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ssl", 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.124 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "false", 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.125 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "threads": { 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threads", 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:58.126 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "32", 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.127 INFO:teuthology.orchestra.run.smithi082.stdout: "username": { 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "username", 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.128 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "username of InfluxDB server user", 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "verify_ssl": { 2023-07-14T19:38:58.129 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "verify_ssl", 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "true", 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.130 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.131 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "insights", 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.132 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.133 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.137 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.139 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.140 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.141 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "iostat", 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.142 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.143 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.144 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.145 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.146 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.147 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.148 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.149 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.150 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "k8sevents", 2023-07-14T19:38:58.151 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_event_retention_days": { 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "ceph_event_retention_days", 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.152 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "7", 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.153 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "config_check_secs": { 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "config_check_secs", 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.154 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "10", 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.156 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.157 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.158 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.159 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.160 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.161 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.162 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.163 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.164 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "localpool", 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.165 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "failure_domain": { 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "failure_domain", 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "host", 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.166 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "failure domain for any created local pool", 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.167 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.168 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.169 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.169 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.169 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.169 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.169 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.169 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.170 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.172 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.173 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.174 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.175 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.176 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.176 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.176 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.176 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.176 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.176 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.177 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.177 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.177 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.177 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.177 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.178 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.178 INFO:teuthology.orchestra.run.smithi082.stdout: "min_size": { 2023-07-14T19:38:58.178 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "min_size", 2023-07-14T19:38:58.178 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.179 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.179 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.179 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.179 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.180 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.180 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.180 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default min_size for any created local pool", 2023-07-14T19:38:58.180 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-14T19:38:58.180 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.181 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.181 INFO:teuthology.orchestra.run.smithi082.stdout: "num_rep": { 2023-07-14T19:38:58.181 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "num_rep", 2023-07-14T19:38:58.181 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.182 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.182 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.182 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3", 2023-07-14T19:38:58.182 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.182 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.183 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.183 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default replica count for any created local pool", 2023-07-14T19:38:58.183 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.183 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.183 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.183 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.184 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_num": { 2023-07-14T19:38:58.184 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_num", 2023-07-14T19:38:58.184 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.184 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.184 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.185 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "128", 2023-07-14T19:38:58.185 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.185 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.185 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.185 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "default pg_num for any created local pool", 2023-07-14T19:38:58.186 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.186 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.186 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.186 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.186 INFO:teuthology.orchestra.run.smithi082.stdout: "prefix": { 2023-07-14T19:38:58.186 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prefix", 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.187 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "name prefix for any created local pool", 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: "subtree": { 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "subtree", 2023-07-14T19:38:58.188 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "rack", 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-14T19:38:58.189 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.190 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mds_autoscaler", 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.191 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.192 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.193 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.194 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.195 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.196 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.197 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.198 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.199 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.200 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirroring", 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.201 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.202 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.203 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.204 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.205 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.206 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.207 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.208 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.209 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.210 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "nfs", 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.211 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.212 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.213 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.214 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.215 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.216 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.217 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.218 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.219 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.220 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "fail_fs": { 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "fail_fs", 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.221 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.222 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.223 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.225 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.226 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.227 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.228 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.229 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.230 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.231 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator": { 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "orchestrator", 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.232 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "cephadm", 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "rook", 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "test_orchestrator" 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Orchestrator backend", 2023-07-14T19:38:58.233 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_perf_query", 2023-07-14T19:38:58.234 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.235 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.236 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.237 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.238 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.239 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.240 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.241 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.242 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.243 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "osd_support", 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.244 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.245 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.246 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.247 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.248 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.249 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.251 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.252 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.253 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "pg_autoscaler", 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.254 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.255 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.256 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.257 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.258 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.259 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.260 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.261 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.263 INFO:teuthology.orchestra.run.smithi082.stdout: "noautoscale": { 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "noautoscale", 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.264 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "global autoscale flag", 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:58.265 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.266 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "threshold": { 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "threshold", 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.267 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "3.0", 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1.0", 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "scaling threshold", 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-14T19:38:58.268 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "progress", 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.269 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_pg_recovery_event": { 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_pg_recovery_event", 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.270 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.271 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.272 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.273 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.274 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.275 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.276 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.278 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.279 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.280 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.281 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: "max_completed_events": { 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_completed_events", 2023-07-14T19:38:58.282 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "50", 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "number of past completed events to remember", 2023-07-14T19:38:58.283 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "sleep_interval": { 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "sleep_interval", 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.284 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "5", 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "how long the module is going to sleep", 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.285 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "prometheus", 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.286 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "cache": { 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "cache", 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.287 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.288 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "exclude_perf_counters": { 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "exclude_perf_counters", 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.289 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.290 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.291 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.292 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.293 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.294 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.295 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.296 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.298 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools": { 2023-07-14T19:38:58.299 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools", 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.300 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.301 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300", 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.302 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "scrape_interval": { 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "scrape_interval", 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15.0", 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "::", 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.305 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:58.306 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "9283", 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.307 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: "stale_cache_strategy": { 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stale_cache_strategy", 2023-07-14T19:38:58.308 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "log", 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.309 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_behaviour": { 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_behaviour", 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.310 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "default", 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "default", 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "error" 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.311 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "standby_error_status_code": { 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "standby_error_status_code", 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.312 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "500", 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "400", 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "599", 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.313 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2023-07-14T19:38:58.314 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.315 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.316 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.317 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.318 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.319 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.320 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.321 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.322 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.322 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.322 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.322 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.322 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.322 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.323 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_snap_create": { 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_snap_create", 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.324 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10", 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.325 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: "mirror_snapshot_schedule": { 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "mirror_snapshot_schedule", 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.326 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.327 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "trash_purge_schedule": { 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "trash_purge_schedule", 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.328 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "restful", 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "enable_auth": { 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enable_auth", 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.331 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: "key_file": { 2023-07-14T19:38:58.332 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "key_file", 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.333 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.335 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.336 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.337 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.338 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.339 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.340 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.341 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.342 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "server_addr": { 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_addr", 2023-07-14T19:38:58.343 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "server_port": { 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "server_port", 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.345 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.346 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rgw", 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.347 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.348 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.349 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.350 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.351 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.352 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.353 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.354 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.355 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.356 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rook", 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.357 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "drive_group_interval": { 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "drive_group_interval", 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "300.0", 2023-07-14T19:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.359 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.360 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.361 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.362 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.363 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.364 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.365 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.366 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.367 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "storage_class": { 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "storage_class", 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "local", 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.369 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "selftest", 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.371 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.372 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.373 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.374 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.375 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.376 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.377 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.379 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "roption1": { 2023-07-14T19:38:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption1", 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.381 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "roption2": { 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "roption2", 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.382 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "xyz", 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.383 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption1": { 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption1", 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.384 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption2": { 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption2", 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.386 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption3": { 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption3", 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "float", 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.388 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption4": { 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption4", 2023-07-14T19:38:58.389 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.390 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption5": { 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption5", 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.391 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.392 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption6": { 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption6", 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.393 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.394 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "rwoption7": { 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rwoption7", 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "1", 2023-07-14T19:38:58.395 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "42", 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.396 INFO:teuthology.orchestra.run.smithi082.stdout: "testkey": { 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testkey", 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.397 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "testlkey": { 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testlkey", 2023-07-14T19:38:58.398 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.399 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "testnewline": { 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "testnewline", 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.400 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.401 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "allow_m_granularity": { 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "allow_m_granularity", 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.403 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "allow minute scheduled snapshots", 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "dump_on_update": { 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "dump_on_update", 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.405 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "dump database to debug log on update", 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.406 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.407 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.408 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.410 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.411 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.412 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.414 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.415 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "stats", 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.416 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.417 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.418 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.419 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.421 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.422 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.423 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.424 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "status", 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.426 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.427 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.428 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.429 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.430 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.431 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.432 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.434 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.435 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telegraf", 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: "address": { 2023-07-14T19:38:58.436 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "address", 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.437 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:58.438 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "15", 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.440 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.441 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.442 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.443 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.444 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.445 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.447 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.448 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.449 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "telemetry", 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_basic": { 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_basic", 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.450 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.451 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_crash": { 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_crash", 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.452 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_device": { 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_device", 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "True", 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.454 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_ident": { 2023-07-14T19:38:58.455 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_ident", 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.456 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "channel_perf": { 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "channel_perf", 2023-07-14T19:38:58.457 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-14T19:38:58.458 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "contact": { 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "contact", 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.459 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.460 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "description": { 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "description", 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.461 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.462 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "device_url": { 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "device_url", 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-14T19:38:58.463 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.464 INFO:teuthology.orchestra.run.smithi082.stdout: "enabled": { 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "enabled", 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.465 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:58.466 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "24", 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "8", 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: "last_opt_revision": { 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "last_opt_revision", 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.468 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "1", 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.469 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard": { 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard", 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.470 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "leaderboard_description": { 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "leaderboard_description", 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.472 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.473 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.474 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.475 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.476 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.477 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.478 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.479 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.480 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.481 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.482 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "organization": { 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "organization", 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.483 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "proxy": { 2023-07-14T19:38:58.484 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "proxy", 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.485 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "url": { 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "url", 2023-07-14T19:38:58.486 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.487 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.488 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "test_orchestrator", 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.489 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.490 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.492 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.493 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.494 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.495 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.496 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.497 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.498 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.499 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.500 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.501 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.502 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.503 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.504 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.505 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.506 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.507 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "max_concurrent_clones": { 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "max_concurrent_clones", 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.508 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "4", 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.509 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "snapshot_clone_delay": { 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snapshot_clone_delay", 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.510 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "0", 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.511 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix", 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: "can_run": true, 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: "error_string": "", 2023-07-14T19:38:58.512 INFO:teuthology.orchestra.run.smithi082.stdout: "module_options": { 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "discovery_interval": { 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "discovery_interval", 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "uint", 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "100", 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.513 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.514 INFO:teuthology.orchestra.run.smithi082.stdout: "identifier": { 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "identifier", 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.515 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "interval": { 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "interval", 2023-07-14T19:38:58.516 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "secs", 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "60", 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.517 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "log_level": { 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_level", 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.518 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.519 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.520 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster": { 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster", 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.521 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.522 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_cluster_level": { 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_cluster_level", 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "info", 2023-07-14T19:38:58.523 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [ 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "", 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "critical", 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "debug", 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "error", 2023-07-14T19:38:58.524 INFO:teuthology.orchestra.run.smithi082.stdout: "info", 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: "warning" 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.525 INFO:teuthology.orchestra.run.smithi082.stdout: "log_to_file": { 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "log_to_file", 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "bool", 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 1, 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "False", 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.526 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_host": { 2023-07-14T19:38:58.527 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_host", 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "", 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.528 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_port": { 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_port", 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "int", 2023-07-14T19:38:58.529 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "10051", 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.530 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "zabbix_sender": { 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "zabbix_sender", 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "str", 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "level": "advanced", 2023-07-14T19:38:58.531 INFO:teuthology.orchestra.run.smithi082.stdout: "flags": 0, 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "min": "", 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "max": "", 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "enum_allowed": [], 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "desc": "", 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "long_desc": "", 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "tags": [], 2023-07-14T19:38:58.532 INFO:teuthology.orchestra.run.smithi082.stdout: "see_also": [] 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: "services": { 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: "dashboard": "https://172.21.15.82:8443/" 2023-07-14T19:38:58.533 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "always_on_modules": { 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "octopus": [ 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2023-07-14T19:38:58.534 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "pacific": [ 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2023-07-14T19:38:58.535 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2023-07-14T19:38:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "quincy": [ 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2023-07-14T19:38:58.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "reef": [ 2023-07-14T19:38:58.538 INFO:teuthology.orchestra.run.smithi082.stdout: "balancer", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "crash", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "devicehealth", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "orchestrator", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "pg_autoscaler", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "progress", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "rbd_support", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "status", 2023-07-14T19:38:58.539 INFO:teuthology.orchestra.run.smithi082.stdout: "telemetry", 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: "volumes" 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: "last_failure_osd_epoch": 84, 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: "active_clients": [ 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.540 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "devicehealth", 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1541840178 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "libcephsqlite", 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3997636669 2023-07-14T19:38:58.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "rbd_support", 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.543 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 240018718 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2023-07-14T19:38:58.544 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2919079198 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.545 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "snap_schedule", 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2995310814 2023-07-14T19:38:58.546 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.547 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:38:58.547 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.547 INFO:teuthology.orchestra.run.smithi082.stdout: "name": "volumes", 2023-07-14T19:38:58.547 INFO:teuthology.orchestra.run.smithi082.stdout: "addrvec": [ 2023-07-14T19:38:58.547 INFO:teuthology.orchestra.run.smithi082.stdout: { 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v2", 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "172.21.15.82:0", 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3863232190 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-07-14T19:38:58.548 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:38:58.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:56 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/646696142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:38:58.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:56 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1596741830' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T19:38:58.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:57 smithi161 ceph-mon[129184]: pgmap v325: 97 pgs: 97 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 460 op/s 2023-07-14T19:38:58.551 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:56 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/646696142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:38:58.551 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:56 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1596741830' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T19:38:58.551 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:57 smithi133 ceph-mon[128894]: pgmap v325: 97 pgs: 97 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 460 op/s 2023-07-14T19:38:58.551 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa2bc0a1300 msgr2=0x7fa2bc0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa2bc0a1300 0x7fa2bc0a37b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0005950 tx=0x7fa2c00058e0 comp rx=0 tx=0).stop 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 msgr2=0x7fa2d019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 0x7fa2d019a6b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8007ee0 tx=0x7fa2c800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 shutdown_connections 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fa2bc0a1300 0x7fa2bc0a37b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2d0104ed0 0x7fa2d019a6b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:58.552 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2d00ffe90 0x7fa2d0195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:58.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 --2- 172.21.15.82:0/1596741830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2d0100830 0x7fa2d01962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:38:58.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 >> 172.21.15.82:0/1596741830 conn(0x7fa2d00fb760 msgr2=0x7fa2d00fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:38:58.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 shutdown_connections 2023-07-14T19:38:58.553 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:38:56.208+0000 7fa2d7e6b700 1 -- 172.21.15.82:0/1596741830 wait complete. 2023-07-14T19:38:58.562 DEBUG:tasks.check_counter:Getting stats from a 2023-07-14T19:38:58.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-14T19:38:58.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:56 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/646696142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:38:58.563 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:56 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1596741830' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-14T19:38:58.564 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:57 smithi082 ceph-mon[127461]: pgmap v325: 97 pgs: 97 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 460 op/s 2023-07-14T19:38:59.023 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:38:59.839 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:38:59 smithi082 ceph-mon[127461]: pgmap v326: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2023-07-14T19:38:59.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:38:59 smithi133 ceph-mon[128894]: pgmap v326: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2023-07-14T19:39:00.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:38:59 smithi161 ceph-mon[129184]: pgmap v326: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2023-07-14T19:39:00.347 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:39:00.390 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 206, 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 197, 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1281712, 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 112048, 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2023-07-14T19:39:00.391 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.035857159, 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011043566, 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.025444124, 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004875864, 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 196, 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.025394994, 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000129566 2023-07-14T19:39:00.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1281712, 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 112048 2023-07-14T19:39:00.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 6, 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 153, 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3264, 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 178496, 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:00.394 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.011166730, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.007226417, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.002175849, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 151, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.038592560, 2023-07-14T19:39:00.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000255579 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-14T19:39:00.396 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 178496 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2023-07-14T19:39:00.397 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000394247, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000310297, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:00.398 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.399 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-07-14T19:39:00.400 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:00.401 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:00.402 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:00.403 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:00.404 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148977, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2023-07-14T19:39:00.405 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:00.406 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2023-07-14T19:39:00.407 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-07-14T19:39:00.408 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 208, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1252652, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 208, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 208, 2023-07-14T19:39:00.409 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1252652, 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:00.410 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:39:00.785 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-14T19:39:00.786 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-14T19:39:00.786 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-07-14T19:39:00.786 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-07-14T19:39:00.786 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-07-14T19:39:00.786 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-07-14T19:39:00.787 DEBUG:tasks.check_counter:Getting stats from d 2023-07-14T19:39:00.787 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-14T19:39:01.341 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:39:01.865 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:01 smithi082 ceph-mon[127461]: pgmap v327: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 331 op/s 2023-07-14T19:39:01.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:01 smithi133 ceph-mon[128894]: pgmap v327: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 331 op/s 2023-07-14T19:39:02.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:01 smithi161 ceph-mon[129184]: pgmap v327: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 331 op/s 2023-07-14T19:39:02.535 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:39:02.535 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:02.535 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 4, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 4, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 2944, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 448, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001816764, 2023-07-14T19:39:02.536 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000052314, 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000856256, 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001042175, 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000347391 2023-07-14T19:39:02.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T19:39:02.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 211, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 202, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1283072, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 115008, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2023-07-14T19:39:02.539 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.036853699, 2023-07-14T19:39:02.540 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011747514, 2023-07-14T19:39:02.540 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.027606665, 2023-07-14T19:39:02.540 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004901954, 2023-07-14T19:39:02.540 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:02.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 201, 2023-07-14T19:39:02.541 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.042110741, 2023-07-14T19:39:02.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000209506 2023-07-14T19:39:02.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.541 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:02.541 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1283072, 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 115008 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:02.542 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2, 2023-07-14T19:39:02.543 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 154, 2023-07-14T19:39:02.543 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 320, 2023-07-14T19:39:02.543 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 178576, 2023-07-14T19:39:02.543 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:02.543 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.011348080, 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.007610146, 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001808409, 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 151, 2023-07-14T19:39:02.544 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.021715444, 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000143810 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.545 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T19:39:02.546 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 178576 2023-07-14T19:39:02.546 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.546 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-07-14T19:39:02.546 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2023-07-14T19:39:02.546 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-07-14T19:39:02.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.547 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-07-14T19:39:02.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:02.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:02.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:02.547 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:02.548 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:02.548 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:02.548 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:02.548 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:02.548 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:02.548 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:02.549 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:02.550 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:02.550 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:02.550 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:02.550 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:02.550 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:02.551 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:02.552 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:02.552 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-07-14T19:39:02.552 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:02.552 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:02.552 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:02.552 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:02.553 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148977, 2023-07-14T19:39:02.553 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2023-07-14T19:39:02.553 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:02.554 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:02.554 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-07-14T19:39:02.554 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-07-14T19:39:02.554 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:02.555 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:02.555 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:02.555 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:02.555 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2023-07-14T19:39:02.556 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2023-07-14T19:39:02.556 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:02.556 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:02.556 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:02.557 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-07-14T19:39:02.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2023-07-14T19:39:02.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2023-07-14T19:39:02.557 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:02.558 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-07-14T19:39:02.558 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:02.558 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-07-14T19:39:02.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:02.559 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:02.559 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-07-14T19:39:02.559 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-07-14T19:39:02.559 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-07-14T19:39:02.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 213, 2023-07-14T19:39:02.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1253302, 2023-07-14T19:39:02.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:02.560 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 213, 2023-07-14T19:39:02.561 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-07-14T19:39:02.561 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-07-14T19:39:02.561 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 213, 2023-07-14T19:39:02.561 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1253302, 2023-07-14T19:39:02.562 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-07-14T19:39:02.562 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:02.562 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:02.562 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:02.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:02.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:02.563 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:39:03.086 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-14T19:39:03.087 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-14T19:39:03.087 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-14T19:39:03.087 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-14T19:39:03.087 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-14T19:39:03.087 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-07-14T19:39:03.088 DEBUG:tasks.check_counter:Getting stats from g 2023-07-14T19:39:03.088 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-14T19:39:03.641 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:39:03.888 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:03 smithi082 ceph-mon[127461]: pgmap v328: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2023-07-14T19:39:03.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:03 smithi133 ceph-mon[128894]: pgmap v328: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2023-07-14T19:39:04.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:03 smithi161 ceph-mon[129184]: pgmap v328: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 258 op/s 2023-07-14T19:39:04.990 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:39:04.991 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:04.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2023-07-14T19:39:04.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2023-07-14T19:39:04.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2023-07-14T19:39:04.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2023-07-14T19:39:04.991 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001214900, 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000220403, 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:04.992 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:04.993 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2, 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 153, 2023-07-14T19:39:04.994 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 320, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 178400, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.010547749, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.007101356, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001576613, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:04.995 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 150, 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.021467206, 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000143114 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:04.996 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 178400 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 212, 2023-07-14T19:39:04.997 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 201, 2023-07-14T19:39:04.998 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1285200, 2023-07-14T19:39:04.998 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 113680, 2023-07-14T19:39:04.998 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2023-07-14T19:39:04.998 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2023-07-14T19:39:04.998 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.035336746, 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011235647, 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.025878268, 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004829025, 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 201, 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.032877555, 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000163569 2023-07-14T19:39:04.999 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:05.000 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:05.000 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:05.000 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:05.000 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:05.000 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1285200, 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 113680 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:05.001 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:05.002 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:05.003 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:05.004 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:05.005 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148994, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 31, 2023-07-14T19:39:05.006 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:05.007 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2023-07-14T19:39:05.008 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-07-14T19:39:05.009 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-07-14T19:39:05.010 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-07-14T19:39:05.010 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 210, 2023-07-14T19:39:05.010 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1252912, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 210, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 210, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1252912, 2023-07-14T19:39:05.011 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-07-14T19:39:05.012 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:05.012 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:05.012 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:05.012 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:05.012 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:05.012 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:39:05.390 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-14T19:39:05.391 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-14T19:39:05.391 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-07-14T19:39:05.391 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-07-14T19:39:05.391 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-07-14T19:39:05.391 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-07-14T19:39:05.391 DEBUG:tasks.check_counter:Getting stats from j 2023-07-14T19:39:05.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-14T19:39:05.733 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:05 smithi082 ceph-mon[127461]: pgmap v329: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 171 op/s 2023-07-14T19:39:05.949 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.a/config 2023-07-14T19:39:05.952 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:05 smithi133 ceph-mon[128894]: pgmap v329: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 171 op/s 2023-07-14T19:39:06.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:05 smithi161 ceph-mon[129184]: pgmap v329: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 171 op/s 2023-07-14T19:39:07.295 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-07-14T19:39:07.295 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:07.295 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 216, 2023-07-14T19:39:07.295 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 205, 2023-07-14T19:39:07.295 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1286288, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 116048, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.037246215, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.012088651, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.026213180, 2023-07-14T19:39:07.296 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004773677, 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 205, 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.033351494, 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000162690 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:07.297 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1286288, 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 116048 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:07.298 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 156, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 320, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 178928, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.010321152, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.006852784, 2023-07-14T19:39:07.299 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001682795, 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 153, 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.034390366, 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000224773 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:07.300 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 178928 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:07.301 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000433787, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T19:39:07.302 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000361761, 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:07.303 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.304 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:07.305 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:07.306 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:07.307 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:07.308 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:07.309 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 149017, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 31, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-07-14T19:39:07.310 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:07.311 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:07.312 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 214, 2023-07-14T19:39:07.313 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1253432, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 214, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 214, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1253432, 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-07-14T19:39:07.314 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-07-14T19:39:07.315 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-07-14T19:39:07.315 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-07-14T19:39:07.315 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:07.315 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-07-14T19:39:07.315 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-07-14T19:39:07.698 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-14T19:39:07.698 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-14T19:39:07.699 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-14T19:39:07.699 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-14T19:39:07.699 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-14T19:39:07.699 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-07-14T19:39:07.699 DEBUG:tasks.check_counter:Getting stats from b 2023-07-14T19:39:07.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-14T19:39:07.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:07 smithi133 ceph-mon[128894]: pgmap v330: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2023-07-14T19:39:07.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:07 smithi082 ceph-mon[127461]: pgmap v330: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2023-07-14T19:39:08.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:07 smithi161 ceph-mon[129184]: pgmap v330: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2023-07-14T19:39:08.336 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:39:09.571 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T19:39:09.571 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:09.571 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 71600, 2023-07-14T19:39:09.571 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 67291, 2023-07-14T19:39:09.572 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 43213310, 2023-07-14T19:39:09.572 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 212205003, 2023-07-14T19:39:09.572 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 97, 2023-07-14T19:39:09.572 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2023-07-14T19:39:09.572 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 3.750568826, 2023-07-14T19:39:09.573 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 1.422522410, 2023-07-14T19:39:09.573 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 1.919689960, 2023-07-14T19:39:09.573 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.760752463, 2023-07-14T19:39:09.573 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:09.573 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 67289, 2023-07-14T19:39:09.573 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 55.680209969, 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000827478 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 93913, 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013678216, 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000145 2023-07-14T19:39:09.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 42208, 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 1168 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 241129, 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 339053, 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 92847009, 2023-07-14T19:39:09.575 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 324317392, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 12, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 10.940275420, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 5.074685042, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 359.945567804, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 1.073196036, 2023-07-14T19:39:09.576 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 339053, 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 140.495951946, 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000414377 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 345134, 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.053989817, 2023-07-14T19:39:09.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000156 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 82193072, 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 164468176 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 51368, 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 40972, 2023-07-14T19:39:09.578 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 32148139, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 108545825, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 23, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 2.929298835, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 1.076213543, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 2.346210863, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.791298008, 2023-07-14T19:39:09.579 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 40969, 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 63.788691800, 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001556998 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 56390, 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.007331423, 2023-07-14T19:39:09.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000130 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 362432, 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 215584 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2023-07-14T19:39:09.581 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8436, 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 85.851785215, 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.010176835 2023-07-14T19:39:09.582 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 25891, 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 5.132925080, 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000198251 2023-07-14T19:39:09.583 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 100711, 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 99449, 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 99449, 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2539.905024117, 2023-07-14T19:39:09.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.025539774 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 1260, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 111, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 1, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 7666, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 77, 2023-07-14T19:39:09.585 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2023-07-14T19:39:09.586 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 25375, 2023-07-14T19:39:09.586 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 17688, 2023-07-14T19:39:09.586 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 7586, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 101, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 104, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 56930, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 1, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 2, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 5, 2023-07-14T19:39:09.587 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 174760, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 150775, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 952, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 1208, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 26, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 1140, 2023-07-14T19:39:09.588 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 364, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 187, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 39832, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 311, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 44381, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 20, 2023-07-14T19:39:09.589 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 246, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 20, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T19:39:09.590 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 13, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 2, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 11, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 7, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 153, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 18694, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 39970, 2023-07-14T19:39:09.591 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 63, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 54751, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 20905, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 753, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 39144 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.592 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 1, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 1747, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 1080, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 96, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2023-07-14T19:39:09.593 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 15671, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 15575, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2023-07-14T19:39:09.594 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 2, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 187, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 77, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2023-07-14T19:39:09.595 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2023-07-14T19:39:09.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.596 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2023-07-14T19:39:09.596 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 106608, 2023-07-14T19:39:09.596 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 106598, 2023-07-14T19:39:09.596 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 106598, 2023-07-14T19:39:09.596 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 10, 2023-07-14T19:39:09.597 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2023-07-14T19:39:09.597 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 2, 2023-07-14T19:39:09.597 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 191, 2023-07-14T19:39:09.597 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 190, 2023-07-14T19:39:09.597 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 190, 2023-07-14T19:39:09.597 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 2, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 1, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 376656888, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 376693180, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3450, 2023-07-14T19:39:09.598 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 176.193832208, 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.051070676 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 12254, 2023-07-14T19:39:09.599 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 62113, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 49859, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 656, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 1816, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 1160, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 25375, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 105760, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 80385, 2023-07-14T19:39:09.600 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 2, 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 35487, 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 35485, 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 228824, 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 207152 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 125598, 2023-07-14T19:39:09.601 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 4581, 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 100711, 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 414, 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 11724, 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 15659, 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 69.662885703, 2023-07-14T19:39:09.602 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.004448744 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8938, 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 342.704896653, 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.038342458 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.603 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.604 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 42498, 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1765.404055451, 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.041540873 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000184315, 2023-07-14T19:39:09.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000184315 2023-07-14T19:39:09.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.607 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2023-07-14T19:39:09.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2023-07-14T19:39:09.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11, 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.004431214, 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000402837 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 71, 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.876278201, 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.026426453 2023-07-14T19:39:09.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11, 2023-07-14T19:39:09.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 4.664646908, 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.424058809 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4812, 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 45.042153565, 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.009360381 2023-07-14T19:39:09.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.613 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2023-07-14T19:39:09.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 532, 2023-07-14T19:39:09.613 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.852945603, 2023-07-14T19:39:09.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.007242378 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 15667, 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 257.337767942, 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.016425465 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.614 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 97, 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 27.603541589, 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.284572593 2023-07-14T19:39:09.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.617 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2023-07-14T19:39:09.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5, 2023-07-14T19:39:09.617 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.153230728, 2023-07-14T19:39:09.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.030646145 2023-07-14T19:39:09.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.618 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2023-07-14T19:39:09.618 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.618 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.618 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001565810, 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001565810 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.619 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6, 2023-07-14T19:39:09.621 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.017622253, 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002937042 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.622 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 11140, 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 21.578818182, 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001937057 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2023-07-14T19:39:09.623 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2023-07-14T19:39:09.624 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 5, 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 3, 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 2, 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 4704, 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 2352, 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 626 2023-07-14T19:39:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 720, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 720, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:09.626 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:09.627 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:09.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:09.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 1506627, 2023-07-14T19:39:09.630 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 12774, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:09.631 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7744, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 161, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T19:39:09.632 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 47144057, 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 478796, 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2023-07-14T19:39:09.633 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 63157, 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 453968614, 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 31, 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 63157, 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2023-07-14T19:39:09.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 63157, 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 20160.534701451, 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.319212988 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 63157, 2023-07-14T19:39:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 147089 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 63157, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 1004, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 62153, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 6915, 2023-07-14T19:39:09.636 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 33572, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 509, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 3959, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 438, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 77, 2023-07-14T19:39:09.637 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 16271, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 26, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 67144, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2023-07-14T19:39:09.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2023-07-14T19:39:09.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2023-07-14T19:39:09.640 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2023-07-14T19:39:09.641 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 95, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 15, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2023-07-14T19:39:09.642 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 14384, 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 585, 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 3209 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2023-07-14T19:39:09.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 46 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 18, 2023-07-14T19:39:09.644 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 15575, 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:09.645 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 364093, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 125523521, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 364093, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:09.646 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 364093, 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 125523521, 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.647 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:09.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 63157, 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 455914420, 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 63157, 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 455914420, 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:09.650 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 63157, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 63157, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 63157, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 63157, 2023-07-14T19:39:09.651 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2023-07-14T19:39:09.652 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 15575, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1959619, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 15575, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:09.653 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 509, 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1959619, 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.654 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x5642e138a0a0": { 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 60, 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 106608, 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 372498876, 2023-07-14T19:39:09.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 106608, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3450, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 372498816, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:09.656 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:09.657 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:09.657 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.657 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:09.657 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T19:39:09.693 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:09 smithi133 ceph-mon[128894]: pgmap v331: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 120 KiB/s wr, 64 op/s 2023-07-14T19:39:09.977 INFO:tasks.check_counter:Daemon mds.b mds.exported=187 2023-07-14T19:39:09.977 INFO:tasks.check_counter:Daemon mds.b mds.imported=311 2023-07-14T19:39:09.977 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-07-14T19:39:09.978 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2023-07-14T19:39:09.978 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2023-07-14T19:39:09.978 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=77 2023-07-14T19:39:09.978 DEBUG:tasks.check_counter:Getting stats from e 2023-07-14T19:39:09.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-14T19:39:09.983 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:09 smithi082 ceph-mon[127461]: pgmap v331: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 120 KiB/s wr, 64 op/s 2023-07-14T19:39:10.010 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:09 smithi161 ceph-mon[129184]: pgmap v331: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 120 KiB/s wr, 64 op/s 2023-07-14T19:39:10.530 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:39:11.643 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T19:39:11.643 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:11.643 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1783, 2023-07-14T19:39:11.643 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1785, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 298674052, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 554520, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.285310269, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.040745225, 2023-07-14T19:39:11.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.182408945, 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.048964800, 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1783, 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.130878446, 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000073403 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.645 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:11.646 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.646 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.646 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.646 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T19:39:11.646 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 843, 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 807, 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 232777284, 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 300584, 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 6, 2023-07-14T19:39:11.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2023-07-14T19:39:11.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.172538132, 2023-07-14T19:39:11.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.021685874, 2023-07-14T19:39:11.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.135325836, 2023-07-14T19:39:11.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.020048934, 2023-07-14T19:39:11.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:11.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 807, 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.059637184, 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000073899 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.650 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1406048, 2023-07-14T19:39:11.650 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 119520 2023-07-14T19:39:11.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.650 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 785, 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 939, 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 301977067, 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 665081, 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2023-07-14T19:39:11.651 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.216957610, 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.018092740, 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.171701682, 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.019954276, 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 936, 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.064164872, 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000068552 2023-07-14T19:39:11.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T19:39:11.653 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 421568 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2023-07-14T19:39:11.654 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4920, 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.238651760, 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000048506 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2023-07-14T19:39:11.655 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2023-07-14T19:39:11.656 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2023-07-14T19:39:11.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.656 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.656 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.657 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2023-07-14T19:39:11.658 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2023-07-14T19:39:11.659 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 127109, 2023-07-14T19:39:11.659 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 21888, 2023-07-14T19:39:11.659 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 105210, 2023-07-14T19:39:11.659 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 11, 2023-07-14T19:39:11.659 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 95840, 2023-07-14T19:39:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 3361, 2023-07-14T19:39:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2023-07-14T19:39:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2023-07-14T19:39:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 3, 2023-07-14T19:39:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2023-07-14T19:39:11.660 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2023-07-14T19:39:11.661 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2023-07-14T19:39:11.662 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2023-07-14T19:39:11.662 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2023-07-14T19:39:11.662 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2023-07-14T19:39:11.662 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2023-07-14T19:39:11.662 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2023-07-14T19:39:11.662 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2023-07-14T19:39:11.663 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2023-07-14T19:39:11.663 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2023-07-14T19:39:11.663 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2023-07-14T19:39:11.663 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T19:39:11.663 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2023-07-14T19:39:11.664 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T19:39:11.664 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2023-07-14T19:39:11.664 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2023-07-14T19:39:11.664 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T19:39:11.665 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T19:39:11.665 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2023-07-14T19:39:11.665 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2023-07-14T19:39:11.665 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T19:39:11.665 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2023-07-14T19:39:11.666 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T19:39:11.666 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T19:39:11.666 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T19:39:11.666 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2023-07-14T19:39:11.667 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2023-07-14T19:39:11.667 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2023-07-14T19:39:11.667 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2023-07-14T19:39:11.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.668 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2023-07-14T19:39:11.668 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2023-07-14T19:39:11.668 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2023-07-14T19:39:11.668 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2023-07-14T19:39:11.668 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2023-07-14T19:39:11.669 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2023-07-14T19:39:11.669 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2023-07-14T19:39:11.669 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2023-07-14T19:39:11.669 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2023-07-14T19:39:11.670 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2023-07-14T19:39:11.670 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2023-07-14T19:39:11.670 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2023-07-14T19:39:11.670 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2023-07-14T19:39:11.670 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2023-07-14T19:39:11.671 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2023-07-14T19:39:11.671 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2023-07-14T19:39:11.671 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2023-07-14T19:39:11.671 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2023-07-14T19:39:11.671 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T19:39:11.672 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2023-07-14T19:39:11.672 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2023-07-14T19:39:11.672 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2023-07-14T19:39:11.672 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2023-07-14T19:39:11.673 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2023-07-14T19:39:11.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.673 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2023-07-14T19:39:11.673 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2023-07-14T19:39:11.673 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2023-07-14T19:39:11.674 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2023-07-14T19:39:11.674 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 0, 2023-07-14T19:39:11.674 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2023-07-14T19:39:11.674 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2023-07-14T19:39:11.674 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2023-07-14T19:39:11.675 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2023-07-14T19:39:11.675 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2023-07-14T19:39:11.675 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 135, 2023-07-14T19:39:11.675 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2023-07-14T19:39:11.676 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2023-07-14T19:39:11.676 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 394268443, 2023-07-14T19:39:11.676 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 0, 2023-07-14T19:39:11.676 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 834123034, 2023-07-14T19:39:11.676 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2023-07-14T19:39:11.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.677 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.677 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 207017 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 45398, 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 107613, 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 62215, 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 1473, 2023-07-14T19:39:11.678 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 2931, 2023-07-14T19:39:11.679 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 1458, 2023-07-14T19:39:11.679 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 127109, 2023-07-14T19:39:11.679 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 184948, 2023-07-14T19:39:11.679 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 57839, 2023-07-14T19:39:11.679 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 278340, 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 207152 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2023-07-14T19:39:11.680 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.681 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2023-07-14T19:39:11.682 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.684 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2023-07-14T19:39:11.684 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.684 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.684 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.685 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.686 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.687 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.688 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.689 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2023-07-14T19:39:11.690 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.691 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.692 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2023-07-14T19:39:11.692 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.692 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.692 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.693 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.694 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.695 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.696 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.697 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2023-07-14T19:39:11.698 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2023-07-14T19:39:11.700 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2023-07-14T19:39:11.701 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 4, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 3, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 3655 2023-07-14T19:39:11.702 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:11.703 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:11.704 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:11.705 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:11.706 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 1813651, 2023-07-14T19:39:11.707 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 46400, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:11.708 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7744, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 161, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T19:39:11.709 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 193204378, 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 1857460, 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.710 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 3148, 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 3148, 2023-07-14T19:39:11.711 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3148, 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 17.997687078, 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005717181 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2023-07-14T19:39:11.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3148, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3149 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 3148, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 3148, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2023-07-14T19:39:11.713 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 1813, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1333, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2023-07-14T19:39:11.714 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2023-07-14T19:39:11.715 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2023-07-14T19:39:11.716 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2023-07-14T19:39:11.717 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2023-07-14T19:39:11.718 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 96, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 15, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 11, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 11, 2023-07-14T19:39:11.719 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2023-07-14T19:39:11.720 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2023-07-14T19:39:11.721 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2023-07-14T19:39:11.722 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T19:39:11.722 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2023-07-14T19:39:11.722 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2023-07-14T19:39:11.722 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2023-07-14T19:39:11.722 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2023-07-14T19:39:11.722 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 3407, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 833105721, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:11.723 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 3407, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3407, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 833105721, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.724 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:11.725 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3148, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 831161385, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3148, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 831161385, 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:11.726 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.727 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.727 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.727 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.727 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.727 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2023-07-14T19:39:11.727 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3148, 2023-07-14T19:39:11.728 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 3148, 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3148, 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3148, 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.729 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:11.730 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.731 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x56328132c460": { 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:11.732 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2023-07-14T19:39:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:11.734 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:11.734 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:11.734 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:11.734 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.734 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:11.734 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T19:39:11.735 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:11 smithi133 ceph-mon[128894]: pgmap v332: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-07-14T19:39:11.735 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:11 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:39:11.986 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:11 smithi082 ceph-mon[127461]: pgmap v332: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-07-14T19:39:11.987 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:11 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:39:12.012 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:11 smithi161 ceph-mon[129184]: pgmap v332: 97 pgs: 97 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-07-14T19:39:12.012 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:11 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:39:12.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:12 smithi133 ceph-mon[128894]: osdmap e96: 12 total, 12 up, 12 in 2023-07-14T19:39:12.984 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:12 smithi082 ceph-mon[127461]: osdmap e96: 12 total, 12 up, 12 in 2023-07-14T19:39:13.013 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:12 smithi161 ceph-mon[129184]: osdmap e96: 12 total, 12 up, 12 in 2023-07-14T19:39:13.515 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-14T19:39:13.516 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-14T19:39:13.516 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-07-14T19:39:13.516 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-07-14T19:39:13.516 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T19:39:13.517 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-07-14T19:39:13.517 DEBUG:tasks.check_counter:Getting stats from h 2023-07-14T19:39:13.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-14T19:39:13.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:13 smithi133 ceph-mon[128894]: pgmap v334: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 25 op/s 2023-07-14T19:39:13.986 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:13 smithi082 ceph-mon[127461]: pgmap v334: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 25 op/s 2023-07-14T19:39:14.012 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:13 smithi161 ceph-mon[129184]: pgmap v334: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 25 op/s 2023-07-14T19:39:14.497 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:39:15.941 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:15 smithi133 ceph-mon[128894]: pgmap v335: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-07-14T19:39:15.985 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:15 smithi082 ceph-mon[127461]: pgmap v335: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-07-14T19:39:16.015 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:15 smithi161 ceph-mon[129184]: pgmap v335: 97 pgs: 14 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 75 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 35 KiB/s wr, 10 op/s 2023-07-14T19:39:16.603 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T19:39:16.604 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:16.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 293770, 2023-07-14T19:39:16.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 492001, 2023-07-14T19:39:16.604 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 125232933, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 534814371, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 15, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 16.494244461, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 7.997044007, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 277.947237432, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.687737065, 2023-07-14T19:39:16.605 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 491999, 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 89.680531808, 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000182277 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 553810, 2023-07-14T19:39:16.606 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.094538658, 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000170 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 93032944, 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 245188480 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 45712, 2023-07-14T19:39:16.607 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 55492, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 36540727, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 319763990, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 11, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 3.700251712, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 1.601984585, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 1.245772050, 2023-07-14T19:39:16.608 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.954204854, 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 55491, 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 42.899290097, 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000773085 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 48510, 2023-07-14T19:39:16.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013457871, 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000277 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 116224 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:16.610 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 12959, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 13122, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 9933866, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 285145925, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 13, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 1.678529651, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.741556358, 2023-07-14T19:39:16.611 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.211993243, 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.665548628, 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 13120, 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 7.192068847, 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000548175 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.612 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 59, 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000010067, 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000170 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3360, 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 320080 2023-07-14T19:39:16.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2023-07-14T19:39:16.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 8309, 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 160.845958715, 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.019358040 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2023-07-14T19:39:16.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 51349, 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 14.436752559, 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000281149 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 155267, 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 145995, 2023-07-14T19:39:16.616 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 145995, 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3634.460614259, 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.024894418 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 9269, 2023-07-14T19:39:16.617 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 356, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 1834, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 158, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 4, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 78657, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 13970, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 64584, 2023-07-14T19:39:16.618 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 103, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 58141, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 66913, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 2, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 2, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 5, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 231258, 2023-07-14T19:39:16.619 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 203447, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 9088, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 472, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 74, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 374, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 15, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2023-07-14T19:39:16.620 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 350, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 47674, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 318, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 53398, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 3, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2023-07-14T19:39:16.621 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 246, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 3, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 2, 2023-07-14T19:39:16.622 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 2, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 1, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 429, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 22290, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 78162, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 5, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 506, 2023-07-14T19:39:16.623 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 75671, 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 40153, 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 1439, 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 74363 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2023-07-14T19:39:16.624 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 848, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 1192, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 211, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2023-07-14T19:39:16.625 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 32842, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 32631, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2023-07-14T19:39:16.626 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 352, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 162, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2023-07-14T19:39:16.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 207018, 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 115273, 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 115273, 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 91745, 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2023-07-14T19:39:16.628 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 940, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 292, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 164, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 164, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 129, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 1, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 394268443, 2023-07-14T19:39:16.629 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 834135673, 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4698, 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 503.141968626, 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.107097055 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.630 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 45129, 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 108673, 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 63544, 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 1491, 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 2818, 2023-07-14T19:39:16.631 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 1327, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 78657, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 191325, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 112668, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 2, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 60414, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 60412, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 297356, 2023-07-14T19:39:16.632 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 198960 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 165142, 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 3980, 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 155267, 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 428, 2023-07-14T19:39:16.633 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 8191, 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 36460, 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 193.524108386, 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005307847 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2449, 2023-07-14T19:39:16.634 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 266.334868988, 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.108752498 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.635 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 33753, 2023-07-14T19:39:16.636 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2696.215280828, 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.079880759 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.637 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.638 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2023-07-14T19:39:16.639 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.640 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 253, 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 6.042775870, 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.023884489 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2023-07-14T19:39:16.641 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7662, 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 44.139638989, 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005760850 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2023-07-14T19:39:16.643 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1424, 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3.653977981, 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002565995 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 36062, 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 345.141799055, 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.009570789 2023-07-14T19:39:16.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2023-07-14T19:39:16.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 215, 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 33.381066943, 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.155260776 2023-07-14T19:39:16.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.647 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5, 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.080834587, 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.016166917 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.648 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.649 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2023-07-14T19:39:16.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.002186394, 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000728798 2023-07-14T19:39:16.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 27709, 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 45.944076238, 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001658092 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2023-07-14T19:39:16.652 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2023-07-14T19:39:16.653 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 4, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 2, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 22794, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 11397, 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 625 2023-07-14T19:39:16.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 120, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 120, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:16.655 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:16.656 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:16.657 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:16.658 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 3488717, 2023-07-14T19:39:16.659 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 43622, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:16.660 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 7864, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 164, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T19:39:16.661 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 159930320, 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 1592727, 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2023-07-14T19:39:16.662 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 47792, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 872958042, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 28, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 47792, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 47792, 2023-07-14T19:39:16.663 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1166.308719216, 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.024403848 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 47792, 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 60919 2023-07-14T19:39:16.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 47792, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 2003, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 45789, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 388, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 4902, 2023-07-14T19:39:16.665 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 696, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 5394, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 409, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 158, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 32967, 2023-07-14T19:39:16.666 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2023-07-14T19:39:16.667 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2023-07-14T19:39:16.667 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2023-07-14T19:39:16.667 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 6, 2023-07-14T19:39:16.667 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 9804, 2023-07-14T19:39:16.667 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2023-07-14T19:39:16.667 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2023-07-14T19:39:16.668 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 5, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2023-07-14T19:39:16.669 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2023-07-14T19:39:16.670 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 96, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 15, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 12, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2023-07-14T19:39:16.671 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 3883, 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 1660, 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 647 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2023-07-14T19:39:16.672 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 139 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2023-07-14T19:39:16.673 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 10, 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 32631, 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:16.674 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:16.675 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:16.675 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 352437, 2023-07-14T19:39:16.675 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 128634482, 2023-07-14T19:39:16.675 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:16.675 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 352437, 2023-07-14T19:39:16.675 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 352437, 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 128634482, 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:16.677 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 47792, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 877324422, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 47792, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 877324422, 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:16.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2023-07-14T19:39:16.679 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 47792, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 47792, 2023-07-14T19:39:16.680 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 47792, 2023-07-14T19:39:16.681 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 47792, 2023-07-14T19:39:16.681 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:16.681 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.681 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.681 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.681 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 32631, 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 4403891, 2023-07-14T19:39:16.682 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 32631, 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 696, 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 4403891, 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:16.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.684 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.684 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.684 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:16.684 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55b97d68e0a0": { 2023-07-14T19:39:16.684 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 574, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 207018, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 829941369, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 207018, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:16.685 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4698, 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 829940795, 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.686 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:16.687 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T19:39:17.110 INFO:tasks.check_counter:Daemon mds.h mds.exported=350 2023-07-14T19:39:17.110 INFO:tasks.check_counter:Daemon mds.h mds.imported=318 2023-07-14T19:39:17.111 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2023-07-14T19:39:17.111 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-14T19:39:17.111 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T19:39:17.111 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=158 2023-07-14T19:39:17.111 DEBUG:tasks.check_counter:Getting stats from k 2023-07-14T19:39:17.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-14T19:39:17.868 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.b/config 2023-07-14T19:39:17.953 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:17 smithi133 ceph-mon[128894]: pgmap v336: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-07-14T19:39:17.987 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:17 smithi082 ceph-mon[127461]: pgmap v336: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-07-14T19:39:18.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:17 smithi161 ceph-mon[129184]: pgmap v336: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-07-14T19:39:19.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:19 smithi133 ceph-mon[128894]: pgmap v337: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-07-14T19:39:19.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:19 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:39:19.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:19 smithi082 ceph-mon[127461]: pgmap v337: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-07-14T19:39:19.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:19 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:39:19.511 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:19 smithi161 ceph-mon[129184]: pgmap v337: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-07-14T19:39:19.511 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:19 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-14T19:39:20.992 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-07-14T19:39:20.992 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:20.992 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-07-14T19:39:20.993 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-07-14T19:39:20.993 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-07-14T19:39:20.993 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-07-14T19:39:20.993 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-07-14T19:39:20.993 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.001294394, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000419743, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:20.994 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:20.995 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 6, 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 161, 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3264, 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 179904, 2023-07-14T19:39:20.996 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.008843124, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.005062315, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002149267, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 159, 2023-07-14T19:39:20.997 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.020335072, 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000127893 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:20.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:20.999 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-14T19:39:20.999 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 179904 2023-07-14T19:39:20.999 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:20.999 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:20.999 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 216, 2023-07-14T19:39:20.999 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 207, 2023-07-14T19:39:21.000 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1284432, 2023-07-14T19:39:21.000 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 117968, 2023-07-14T19:39:21.000 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-07-14T19:39:21.000 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-07-14T19:39:21.000 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.036710799, 2023-07-14T19:39:21.000 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.010310443, 2023-07-14T19:39:21.001 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.028529757, 2023-07-14T19:39:21.001 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.005531879, 2023-07-14T19:39:21.001 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:21.002 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 206, 2023-07-14T19:39:21.002 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.017952976, 2023-07-14T19:39:21.002 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000087150 2023-07-14T19:39:21.002 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:21.002 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:21.003 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:21.003 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:21.003 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:21.003 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:21.004 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1284432, 2023-07-14T19:39:21.004 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 117968 2023-07-14T19:39:21.004 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:21.004 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-07-14T19:39:21.005 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-07-14T19:39:21.005 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-07-14T19:39:21.005 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:21.005 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-07-14T19:39:21.005 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:21.005 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:21.006 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:21.007 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:21.007 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:21.007 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:21.007 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:21.007 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:21.008 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:21.009 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:21.010 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:21.010 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:21.010 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-07-14T19:39:21.010 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:21.010 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:21.010 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 149057, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 32, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-07-14T19:39:21.011 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:21.012 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:21.013 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-07-14T19:39:21.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-07-14T19:39:21.014 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:21.014 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-07-14T19:39:21.014 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-07-14T19:39:21.014 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-07-14T19:39:21.014 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 218, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1253952, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 218, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 218, 2023-07-14T19:39:21.015 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1253952, 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-07-14T19:39:21.016 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-07-14T19:39:21.453 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:21 smithi133 ceph-mon[128894]: pgmap v338: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-07-14T19:39:21.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:21 smithi082 ceph-mon[127461]: pgmap v338: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-07-14T19:39:21.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:21 smithi161 ceph-mon[129184]: pgmap v338: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-07-14T19:39:21.514 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-14T19:39:21.514 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-14T19:39:21.514 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-07-14T19:39:21.515 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-07-14T19:39:21.515 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-07-14T19:39:21.515 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-07-14T19:39:21.515 DEBUG:tasks.check_counter:Getting stats from c 2023-07-14T19:39:21.516 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-14T19:39:22.270 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:39:22.978 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: pgmap v339: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:22.979 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:22.980 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:22.980 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-07-14T19:39:22.980 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:22 smithi161 ceph-mon[129184]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-07-14T19:39:23.203 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.204 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.204 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: pgmap v339: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-07-14T19:39:23.204 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-07-14T19:39:23.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:22 smithi133 ceph-mon[128894]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-07-14T19:39:23.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: pgmap v339: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-07-14T19:39:23.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:23.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:23.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-07-14T19:39:23.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:22 smithi082 ceph-mon[127461]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-07-14T19:39:24.125 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T19:39:24.126 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:24.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 442, 2023-07-14T19:39:24.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 444, 2023-07-14T19:39:24.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 28716904, 2023-07-14T19:39:24.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 137472, 2023-07-14T19:39:24.126 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.053594322, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.011680317, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.023430128, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.014041847, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 442, 2023-07-14T19:39:24.127 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.048627583, 2023-07-14T19:39:24.128 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000110017 2023-07-14T19:39:24.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.128 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:24.129 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.129 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.129 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.129 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.130 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-14T19:39:24.130 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2023-07-14T19:39:24.130 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.130 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:24.131 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 319, 2023-07-14T19:39:24.131 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 313, 2023-07-14T19:39:24.131 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 45574854, 2023-07-14T19:39:24.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 109892, 2023-07-14T19:39:24.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:24.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2023-07-14T19:39:24.132 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.050114615, 2023-07-14T19:39:24.133 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.007752447, 2023-07-14T19:39:24.133 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.034985709, 2023-07-14T19:39:24.133 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.007555707, 2023-07-14T19:39:24.134 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:24.134 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 313, 2023-07-14T19:39:24.134 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.023287258, 2023-07-14T19:39:24.135 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000074400 2023-07-14T19:39:24.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.135 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:24.135 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.136 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.136 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.136 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.136 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 189424, 2023-07-14T19:39:24.137 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 36128 2023-07-14T19:39:24.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.137 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:24.137 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 149, 2023-07-14T19:39:24.137 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 193, 2023-07-14T19:39:24.138 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 12626631, 2023-07-14T19:39:24.138 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 203590, 2023-07-14T19:39:24.138 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 6, 2023-07-14T19:39:24.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2023-07-14T19:39:24.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.021932952, 2023-07-14T19:39:24.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005716391, 2023-07-14T19:39:24.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.009557311, 2023-07-14T19:39:24.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.005105044, 2023-07-14T19:39:24.139 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 191, 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.034863417, 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000182530 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.140 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 157552 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2023-07-14T19:39:24.141 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1303, 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.151696275, 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000116420 2023-07-14T19:39:24.142 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.143 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.144 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2023-07-14T19:39:24.144 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2023-07-14T19:39:24.144 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2023-07-14T19:39:24.144 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2023-07-14T19:39:24.144 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.145 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.145 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.145 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2023-07-14T19:39:24.145 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2023-07-14T19:39:24.147 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2023-07-14T19:39:24.147 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2023-07-14T19:39:24.148 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2023-07-14T19:39:24.148 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2023-07-14T19:39:24.148 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2023-07-14T19:39:24.149 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 20512, 2023-07-14T19:39:24.149 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 14316, 2023-07-14T19:39:24.149 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 6193, 2023-07-14T19:39:24.149 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 3, 2023-07-14T19:39:24.150 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 60, 2023-07-14T19:39:24.150 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 424, 2023-07-14T19:39:24.150 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2023-07-14T19:39:24.150 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2023-07-14T19:39:24.151 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 3, 2023-07-14T19:39:24.151 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2023-07-14T19:39:24.151 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2023-07-14T19:39:24.151 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2023-07-14T19:39:24.152 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2023-07-14T19:39:24.152 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2023-07-14T19:39:24.152 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2023-07-14T19:39:24.152 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2023-07-14T19:39:24.153 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2023-07-14T19:39:24.153 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2023-07-14T19:39:24.153 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2023-07-14T19:39:24.153 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2023-07-14T19:39:24.154 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2023-07-14T19:39:24.154 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2023-07-14T19:39:24.154 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2023-07-14T19:39:24.154 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2023-07-14T19:39:24.155 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2023-07-14T19:39:24.155 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2023-07-14T19:39:24.155 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 246, 2023-07-14T19:39:24.155 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 6, 2023-07-14T19:39:24.156 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T19:39:24.156 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2023-07-14T19:39:24.156 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T19:39:24.156 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2023-07-14T19:39:24.157 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2023-07-14T19:39:24.157 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T19:39:24.157 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T19:39:24.157 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2023-07-14T19:39:24.158 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2023-07-14T19:39:24.158 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T19:39:24.158 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2023-07-14T19:39:24.159 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T19:39:24.159 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T19:39:24.159 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T19:39:24.159 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2023-07-14T19:39:24.160 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2023-07-14T19:39:24.160 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2023-07-14T19:39:24.160 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2023-07-14T19:39:24.160 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.161 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2023-07-14T19:39:24.161 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2023-07-14T19:39:24.161 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2023-07-14T19:39:24.161 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2023-07-14T19:39:24.162 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2023-07-14T19:39:24.162 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2023-07-14T19:39:24.162 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2023-07-14T19:39:24.162 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2023-07-14T19:39:24.163 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2023-07-14T19:39:24.163 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2023-07-14T19:39:24.163 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2023-07-14T19:39:24.163 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2023-07-14T19:39:24.164 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2023-07-14T19:39:24.164 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2023-07-14T19:39:24.164 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2023-07-14T19:39:24.164 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2023-07-14T19:39:24.165 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2023-07-14T19:39:24.165 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2023-07-14T19:39:24.165 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T19:39:24.165 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2023-07-14T19:39:24.166 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2023-07-14T19:39:24.166 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2023-07-14T19:39:24.166 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2023-07-14T19:39:24.166 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2023-07-14T19:39:24.167 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.167 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2023-07-14T19:39:24.167 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2023-07-14T19:39:24.168 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2023-07-14T19:39:24.168 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2023-07-14T19:39:24.168 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 0, 2023-07-14T19:39:24.168 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2023-07-14T19:39:24.169 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2023-07-14T19:39:24.169 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2023-07-14T19:39:24.169 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2023-07-14T19:39:24.169 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2023-07-14T19:39:24.170 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 2, 2023-07-14T19:39:24.170 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2023-07-14T19:39:24.170 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2023-07-14T19:39:24.170 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 376656888, 2023-07-14T19:39:24.171 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 0, 2023-07-14T19:39:24.171 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 376693292, 2023-07-14T19:39:24.171 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2023-07-14T19:39:24.171 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.172 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.172 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.172 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.172 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 24376 2023-07-14T19:39:24.173 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.173 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2023-07-14T19:39:24.173 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 7044, 2023-07-14T19:39:24.173 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 14741, 2023-07-14T19:39:24.174 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 7697, 2023-07-14T19:39:24.174 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 342, 2023-07-14T19:39:24.174 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 460, 2023-07-14T19:39:24.174 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 118, 2023-07-14T19:39:24.175 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 20512, 2023-07-14T19:39:24.175 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 22005, 2023-07-14T19:39:24.175 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 1493, 2023-07-14T19:39:24.175 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2023-07-14T19:39:24.176 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2023-07-14T19:39:24.176 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2023-07-14T19:39:24.176 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 86464, 2023-07-14T19:39:24.176 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207128 2023-07-14T19:39:24.177 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.177 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2023-07-14T19:39:24.177 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2023-07-14T19:39:24.178 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2023-07-14T19:39:24.178 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2023-07-14T19:39:24.178 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2023-07-14T19:39:24.178 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2023-07-14T19:39:24.179 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2023-07-14T19:39:24.179 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.179 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.179 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.180 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2023-07-14T19:39:24.180 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.180 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.181 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.181 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.181 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2023-07-14T19:39:24.181 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.182 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.182 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.182 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2023-07-14T19:39:24.183 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.183 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.183 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.183 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.184 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2023-07-14T19:39:24.184 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.184 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.184 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.185 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2023-07-14T19:39:24.185 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.185 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.186 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.186 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2023-07-14T19:39:24.186 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.187 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.187 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.187 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.187 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2023-07-14T19:39:24.188 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.188 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.188 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.189 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2023-07-14T19:39:24.189 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.189 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.190 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.190 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2023-07-14T19:39:24.190 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.190 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.191 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.191 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.191 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2023-07-14T19:39:24.191 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.192 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.192 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.192 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.193 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2023-07-14T19:39:24.193 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.193 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.193 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.194 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.194 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2023-07-14T19:39:24.194 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.194 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.195 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.195 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.195 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2023-07-14T19:39:24.195 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.196 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2023-07-14T19:39:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.197 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2023-07-14T19:39:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.198 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.199 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.199 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.199 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2023-07-14T19:39:24.199 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.200 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.200 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.200 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.200 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2023-07-14T19:39:24.201 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.201 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.201 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.201 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.202 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2023-07-14T19:39:24.202 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.202 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.202 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.203 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.203 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2023-07-14T19:39:24.203 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.203 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.204 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.204 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.204 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2023-07-14T19:39:24.204 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.205 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.205 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.205 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.205 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2023-07-14T19:39:24.206 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.206 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.206 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.207 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.207 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2023-07-14T19:39:24.207 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.207 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.208 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.208 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2023-07-14T19:39:24.208 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.209 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.209 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.209 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.209 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2023-07-14T19:39:24.210 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.210 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.210 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.210 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.211 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2023-07-14T19:39:24.211 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.211 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.211 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.212 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.212 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2023-07-14T19:39:24.212 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.212 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.213 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.213 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.213 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2023-07-14T19:39:24.213 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.214 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.214 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.214 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.214 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2023-07-14T19:39:24.215 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.215 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.215 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.216 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.216 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2023-07-14T19:39:24.216 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2023-07-14T19:39:24.216 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2023-07-14T19:39:24.217 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.217 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.217 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.218 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.218 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.218 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2023-07-14T19:39:24.218 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2023-07-14T19:39:24.219 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 2, 2023-07-14T19:39:24.219 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 0, 2023-07-14T19:39:24.219 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 0, 2023-07-14T19:39:24.219 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2023-07-14T19:39:24.220 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2023-07-14T19:39:24.220 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2023-07-14T19:39:24.220 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2648 2023-07-14T19:39:24.220 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.221 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T19:39:24.221 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:24.221 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:24.222 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:24.222 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:24.222 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:24.222 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:24.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:24.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:24.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:24.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:24.223 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:24.224 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:24.225 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:24.226 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:24.226 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:24.226 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:24.227 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:24.227 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:24.227 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T19:39:24.227 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:24.228 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:24.228 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:24.228 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:24.228 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 467345, 2023-07-14T19:39:24.229 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 7293, 2023-07-14T19:39:24.229 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:24.229 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:24.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T19:39:24.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T19:39:24.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:24.230 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:24.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:24.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:24.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 7864, 2023-07-14T19:39:24.231 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 164, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 30709509, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 245923, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:24.232 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2023-07-14T19:39:24.233 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 827, 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 26, 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 4, 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 827, 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 827, 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.185976109, 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001434070 2023-07-14T19:39:24.234 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 827, 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 828 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 827, 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 827, 2023-07-14T19:39:24.235 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 513, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 311, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2023-07-14T19:39:24.236 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2023-07-14T19:39:24.237 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 2, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2023-07-14T19:39:24.238 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2023-07-14T19:39:24.239 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2023-07-14T19:39:24.240 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 96, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2023-07-14T19:39:24.241 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 7, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 9, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 9, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2023-07-14T19:39:24.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2023-07-14T19:39:24.243 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2023-07-14T19:39:24.244 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 906, 2023-07-14T19:39:24.245 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 86829965, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 906, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 906, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 86829965, 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:24.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:24.247 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 827, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 86483445, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 827, 2023-07-14T19:39:24.248 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 86483445, 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2023-07-14T19:39:24.249 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:24.250 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 827, 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 827, 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 827, 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 827, 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.251 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:24.252 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.253 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55ac24554460": { 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:24.254 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T19:39:24.255 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:24.256 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:24.256 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:24.256 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:24.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.256 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:24.256 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T19:39:24.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.257 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:23 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.258 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:23 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.259 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi161 to 196.2M 2023-07-14T19:39:24.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: Unable to set osd_memory_target on smithi161 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-07-14T19:39:24.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:24.260 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:23 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:24.655 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-14T19:39:24.656 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-14T19:39:24.656 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-07-14T19:39:24.657 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-14T19:39:24.657 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T19:39:24.657 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-07-14T19:39:24.658 DEBUG:tasks.check_counter:Getting stats from f 2023-07-14T19:39:24.658 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-14T19:39:24.949 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: pgmap v340: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-07-14T19:39:24.949 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.949 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.949 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.949 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:24.950 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: Adjusting osd_memory_target on smithi133 to 196.2M 2023-07-14T19:39:24.950 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: Unable to set osd_memory_target on smithi133 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T19:39:24.950 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:39:24.950 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:39:24.950 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:24 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:25.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: pgmap v340: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-07-14T19:39:25.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.205 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: Adjusting osd_memory_target on smithi133 to 196.2M 2023-07-14T19:39:25.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: Unable to set osd_memory_target on smithi133 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T19:39:25.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:39:25.206 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:39:25.207 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:24 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:25.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: pgmap v340: 97 pgs: 41 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 33 active+clean; 1.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2023-07-14T19:39:25.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-14T19:39:25.235 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: Adjusting osd_memory_target on smithi133 to 196.2M 2023-07-14T19:39:25.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: Unable to set osd_memory_target on smithi133 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-07-14T19:39:25.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-14T19:39:25.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-14T19:39:25.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:24 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' 2023-07-14T19:39:25.348 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:39:27.078 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T19:39:27.078 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:27.078 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 968, 2023-07-14T19:39:27.078 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 968, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 138309965, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 300316, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.141356512, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.018806028, 2023-07-14T19:39:27.079 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.087034474, 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.027712652, 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 967, 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.087767150, 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000090762 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:27.080 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:27.081 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 816, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 972, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 155573623, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 679735, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 9, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 7, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.132898069, 2023-07-14T19:39:27.082 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.023449447, 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.080468935, 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.022041037, 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 969, 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.057723613, 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000059570 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.083 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 426576 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.084 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1320, 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1286, 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 98585881, 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 450789, 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2023-07-14T19:39:27.085 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.182416756, 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.037454612, 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.101894891, 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.041745128, 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1285, 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.087774524, 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000068307 2023-07-14T19:39:27.086 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1404192, 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 121440 2023-07-14T19:39:27.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T19:39:27.088 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4567, 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.142887082, 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000031286 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T19:39:27.089 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2023-07-14T19:39:27.090 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2023-07-14T19:39:27.091 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 47788, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 2700, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 43132, 2023-07-14T19:39:27.092 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 1956, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 43930, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 1126, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2023-07-14T19:39:27.093 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2023-07-14T19:39:27.094 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2023-07-14T19:39:27.095 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T19:39:27.096 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T19:39:27.097 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2023-07-14T19:39:27.098 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2023-07-14T19:39:27.099 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2023-07-14T19:39:27.100 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2023-07-14T19:39:27.101 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2023-07-14T19:39:27.102 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 131, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 82301375, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 0, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 393900237, 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2023-07-14T19:39:27.103 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 105259 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2023-07-14T19:39:27.104 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 13221, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 64471, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 51250, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 458, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 1695, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 1237, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 47788, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 103681, 2023-07-14T19:39:27.105 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 55893, 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 151392, 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 207152 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2023-07-14T19:39:27.106 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2023-07-14T19:39:27.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.111 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.112 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2023-07-14T19:39:27.113 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.114 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2023-07-14T19:39:27.115 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.116 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.117 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.118 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.119 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.120 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.121 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2023-07-14T19:39:27.122 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2023-07-14T19:39:27.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2023-07-14T19:39:27.127 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 6658 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:27.128 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:27.129 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:27.130 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:27.131 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:27.132 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 697236, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 13523, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:27.133 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 7864, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 164, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:27.134 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 64202029, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 614437, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.135 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2837, 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2837, 2023-07-14T19:39:27.136 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2837, 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 18.470625957, 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006510618 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2837, 2023-07-14T19:39:27.137 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2838 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2837, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2837, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1679, 2023-07-14T19:39:27.138 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 1156, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2023-07-14T19:39:27.139 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2023-07-14T19:39:27.140 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2023-07-14T19:39:27.141 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2023-07-14T19:39:27.142 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2023-07-14T19:39:27.143 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 96, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 15, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 11, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 11, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2023-07-14T19:39:27.144 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2023-07-14T19:39:27.145 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2023-07-14T19:39:27.146 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:27.147 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 3100, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 392174209, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 3100, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 3100, 2023-07-14T19:39:27.148 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 392174209, 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2023-07-14T19:39:27.149 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2837, 2023-07-14T19:39:27.150 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 390278677, 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2837, 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 390278677, 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.151 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:27.152 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2837, 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2837, 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2837, 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2837, 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.153 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:27.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T19:39:27.155 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55c72f3c6460": { 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:27.156 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:27.157 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:27.158 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T19:39:27.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:26 smithi161 ceph-mon[129184]: pgmap v341: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T19:39:27.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:26 smithi082 ceph-mon[127461]: pgmap v341: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T19:39:27.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:26 smithi133 ceph-mon[128894]: pgmap v341: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-14T19:39:27.888 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-14T19:39:27.889 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-14T19:39:27.889 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-07-14T19:39:27.889 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-14T19:39:27.889 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T19:39:27.889 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-07-14T19:39:27.889 DEBUG:tasks.check_counter:Getting stats from i 2023-07-14T19:39:27.890 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-14T19:39:28.604 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:39:29.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:29 smithi161 ceph-mon[129184]: pgmap v342: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T19:39:29.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:29 smithi133 ceph-mon[128894]: pgmap v342: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T19:39:29.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:29 smithi082 ceph-mon[127461]: pgmap v342: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-14T19:39:30.363 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T19:39:30.363 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 51467, 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 55827, 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 20055195, 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 144293655, 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 9, 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:30.364 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 2.670857279, 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.126371815, 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.608927509, 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.228649852, 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 55827, 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.483194726, 2023-07-14T19:39:30.365 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000062392 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 91678, 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.012806408, 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000139 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-14T19:39:30.366 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 313072 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 33399, 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 38522, 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 34061623, 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 137342299, 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2023-07-14T19:39:30.367 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 2.209550110, 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.888373524, 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.139852415, 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.365326917, 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 38524, 2023-07-14T19:39:30.368 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.476098515, 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000090232 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 51375, 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.007484166, 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000145 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.369 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1413456, 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 126416 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 246093, 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 361947, 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 96042625, 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 332607052, 2023-07-14T19:39:30.370 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 18, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 10.807117074, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 5.320022881, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 25.351357319, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.543897910, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 361940, 2023-07-14T19:39:30.371 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 34.217654605, 2023-07-14T19:39:30.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000094539 2023-07-14T19:39:30.372 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.372 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:30.372 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 365087, 2023-07-14T19:39:30.372 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.054122514, 2023-07-14T19:39:30.373 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000148 2023-07-14T19:39:30.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 89658368, 2023-07-14T19:39:30.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 175676832 2023-07-14T19:39:30.373 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.373 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9095, 2023-07-14T19:39:30.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 63.945155696, 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007030803 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 31284, 2023-07-14T19:39:30.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 8.410385684, 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000268839 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 101700, 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 98441, 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2023-07-14T19:39:30.376 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 98441, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2533.987513545, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.025741180 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 3259, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 132, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2023-07-14T19:39:30.377 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 877, 2023-07-14T19:39:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 83, 2023-07-14T19:39:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2023-07-14T19:39:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 22681, 2023-07-14T19:39:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 3968, 2023-07-14T19:39:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 10904, 2023-07-14T19:39:30.378 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 7809, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 17009, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 63078, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 2, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 2, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 5, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 173141, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 151414, 2023-07-14T19:39:30.379 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 2890, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 973, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 45, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 1531, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2023-07-14T19:39:30.380 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 539, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 48152, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 447, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 37879, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2023-07-14T19:39:30.381 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 246, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2023-07-14T19:39:30.382 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2023-07-14T19:39:30.383 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2023-07-14T19:39:30.383 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 822, 2023-07-14T19:39:30.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 17010, 2023-07-14T19:39:30.383 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 41509, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 71, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 40356, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 21345, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 821, 2023-07-14T19:39:30.384 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 47459 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 3, 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 2, 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 1049, 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 1777, 2023-07-14T19:39:30.385 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 62, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 19153, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 19091, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2023-07-14T19:39:30.386 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 540, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 83, 2023-07-14T19:39:30.387 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 105259, 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 25906, 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 25906, 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 79353, 2023-07-14T19:39:30.388 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 1024, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 166, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 38, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 38, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 129, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 1, 2023-07-14T19:39:30.389 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 82301375, 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 393905636, 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4415, 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 369.127933012, 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.083607685 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.390 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 11542, 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 65453, 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 53911, 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 485, 2023-07-14T19:39:30.391 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 1745, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 1260, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 22681, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 108598, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 85917, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 2, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 39576, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 39574, 2023-07-14T19:39:30.392 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 211632, 2023-07-14T19:39:30.393 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 198960 2023-07-14T19:39:30.393 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.393 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2023-07-14T19:39:30.393 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 113780, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 8301, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 101700, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 340, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 16219, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15174, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 83.999940996, 2023-07-14T19:39:30.394 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005535781 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 313, 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 11.841042537, 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.037830806 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2023-07-14T19:39:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.396 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 46541, 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2067.868478707, 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.044431114 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2023-07-14T19:39:30.397 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000286185, 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000286185 2023-07-14T19:39:30.398 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.399 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 56, 2023-07-14T19:39:30.401 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.802508221, 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.014330503 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.402 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 5601, 2023-07-14T19:39:30.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 27.409357754, 2023-07-14T19:39:30.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.004893654 2023-07-14T19:39:30.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.404 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2023-07-14T19:39:30.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 673, 2023-07-14T19:39:30.404 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.084165233, 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001610943 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15172, 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 299.770518898, 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.019758141 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.405 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 62, 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 13.508019740, 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.217871286 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2023-07-14T19:39:30.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.409 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.411 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.412 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 14848, 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 27.703195274, 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001865786 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2023-07-14T19:39:30.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.414 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.414 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 6355, 2023-07-14T19:39:30.415 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 6355, 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 670 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:30.416 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:30.417 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:30.418 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:30.419 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:30.420 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 1195264, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 11263, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:30.421 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 7864, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 164, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:30.422 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 43265268, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 471600, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.423 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2023-07-14T19:39:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2023-07-14T19:39:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2023-07-14T19:39:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 26926, 2023-07-14T19:39:30.424 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 410854171, 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 28, 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 26926, 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 26926, 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 600.414847485, 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.022298701 2023-07-14T19:39:30.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 26926, 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 29029 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 26926, 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 982, 2023-07-14T19:39:30.426 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 25944, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 120, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 296, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 627, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 5042, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 329, 2023-07-14T19:39:30.427 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 79, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 19217, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2023-07-14T19:39:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 592, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2023-07-14T19:39:30.429 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2023-07-14T19:39:30.430 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2023-07-14T19:39:30.431 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 96, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 15, 2023-07-14T19:39:30.432 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 1945, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 487, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 294 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.433 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 25 2023-07-14T19:39:30.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2023-07-14T19:39:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2023-07-14T19:39:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 10, 2023-07-14T19:39:30.435 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 10, 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 19091, 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:30.436 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 330955, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 109562399, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 330955, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 330955, 2023-07-14T19:39:30.437 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 109562399, 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.438 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:30.439 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 26926, 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 413257277, 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 26926, 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 413257277, 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.440 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2023-07-14T19:39:30.441 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 26926, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 26926, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 26926, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 26926, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.442 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:30.443 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 19091, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 2412783, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 19091, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 627, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2412783, 2023-07-14T19:39:30.444 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:30.445 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.445 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.445 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.445 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x555d6154c0a0": { 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 105259, 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 389711332, 2023-07-14T19:39:30.446 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 105259, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 4415, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 389711332, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:30.447 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:30.448 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:30.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:30.448 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T19:39:30.852 INFO:tasks.check_counter:Daemon mds.i mds.exported=539 2023-07-14T19:39:30.852 INFO:tasks.check_counter:Daemon mds.i mds.imported=447 2023-07-14T19:39:30.852 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2023-07-14T19:39:30.853 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-07-14T19:39:30.853 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-07-14T19:39:30.853 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=83 2023-07-14T19:39:30.853 DEBUG:tasks.check_counter:Getting stats from l 2023-07-14T19:39:30.853 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81c9387d2e7576da065747ec9c16d1c7d69016ec shell --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-14T19:39:31.263 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:31 smithi161 ceph-mon[129184]: pgmap v343: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T19:39:31.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:31 smithi133 ceph-mon[128894]: pgmap v343: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T19:39:31.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:31 smithi082 ceph-mon[127461]: pgmap v343: 97 pgs: 41 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 34 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-14T19:39:31.700 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/mon.c/config 2023-07-14T19:39:33.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:33 smithi133 ceph-mon[128894]: pgmap v344: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-14T19:39:33.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:33 smithi082 ceph-mon[127461]: pgmap v344: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-14T19:39:33.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:33 smithi161 ceph-mon[129184]: pgmap v344: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2023-07-14T19:39:35.178 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-07-14T19:39:35.178 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 4, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 4, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 2944, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 448, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 4, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 0, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.001603744, 2023-07-14T19:39:35.179 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.000039994, 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.000721079, 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3, 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000866424, 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000288808 2023-07-14T19:39:35.180 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.181 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:35.181 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:35.181 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:35.181 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:35.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.182 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-14T19:39:35.182 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2023-07-14T19:39:35.182 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.182 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2023-07-14T19:39:35.182 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 2, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 162, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 320, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 179984, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.008824465, 2023-07-14T19:39:35.183 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.005646667, 2023-07-14T19:39:35.184 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.001469043, 2023-07-14T19:39:35.184 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-14T19:39:35.184 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:35.184 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 159, 2023-07-14T19:39:35.184 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.016924692, 2023-07-14T19:39:35.185 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000106444 2023-07-14T19:39:35.185 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.185 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:35.185 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:35.185 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 179984 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 220, 2023-07-14T19:39:35.186 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 211, 2023-07-14T19:39:35.187 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 1285520, 2023-07-14T19:39:35.187 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 120336, 2023-07-14T19:39:35.187 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 2, 2023-07-14T19:39:35.187 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 1, 2023-07-14T19:39:35.187 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.037728368, 2023-07-14T19:39:35.187 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.011206296, 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.027857072, 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.005497292, 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 210, 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.010866979, 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000051747 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.188 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1285520, 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 120336 2023-07-14T19:39:35.189 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 0, 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2023-07-14T19:39:35.190 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2023-07-14T19:39:35.191 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2023-07-14T19:39:35.192 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2023-07-14T19:39:35.193 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2023-07-14T19:39:35.194 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 148977, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 30, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2023-07-14T19:39:35.195 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2023-07-14T19:39:35.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2023-07-14T19:39:35.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2023-07-14T19:39:35.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2023-07-14T19:39:35.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2023-07-14T19:39:35.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2023-07-14T19:39:35.196 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 0, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 0, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 928, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 2, 2023-07-14T19:39:35.197 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2023-07-14T19:39:35.198 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 222, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1254472, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 222, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 222, 2023-07-14T19:39:35.199 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1254472, 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout: } 2023-07-14T19:39:35.200 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-07-14T19:39:35.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:35 smithi133 ceph-mon[128894]: pgmap v345: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:39:35.511 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:35 smithi161 ceph-mon[129184]: pgmap v345: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:39:35.697 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-14T19:39:35.697 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-14T19:39:35.697 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-07-14T19:39:35.697 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-07-14T19:39:35.697 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-07-14T19:39:35.698 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-07-14T19:39:35.698 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-14T19:39:35.709 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-14T19:39:35.710 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:39:35.710 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T19:39:35.743 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:39:35.743 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T19:39:35.822 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:39:36.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.063+0000 7fb0fdf64700 1 -- 172.21.15.82:0/2345437213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 msgr2=0x7fb0f8102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.063+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/2345437213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 0x7fb0f8102a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec01a9d0 tx=0x7fb0ec01ace0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.073+0000 7fb0fdf64700 1 -- 172.21.15.82:0/2345437213 shutdown_connections 2023-07-14T19:39:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.073+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/2345437213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0f8103110 0x7fb0f810ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.073+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/2345437213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f80fe260 0x7fb0f80fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.073+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/2345437213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 0x7fb0f8102a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.073+0000 7fb0fdf64700 1 -- 172.21.15.82:0/2345437213 >> 172.21.15.82:0/2345437213 conn(0x7fb0f8074db0 msgr2=0x7fb0f80751b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.079+0000 7fb0fdf64700 1 -- 172.21.15.82:0/2345437213 shutdown_connections 2023-07-14T19:39:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.083+0000 7fb0fdf64700 1 -- 172.21.15.82:0/2345437213 wait complete. 2023-07-14T19:39:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.083+0000 7fb0fdf64700 1 Processor -- start 2023-07-14T19:39:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 -- start start 2023-07-14T19:39:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0f80fe260 0x7fb0f8106020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 0x7fb0f8106560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 0x7fb0f81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f8106de0 con 0x7fb0f8103110 2023-07-14T19:39:36.106 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f8197a00 con 0x7fb0f80fe260 2023-07-14T19:39:36.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0fdf64700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f8197bc0 con 0x7fb0f80fec00 2023-07-14T19:39:36.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 0x7fb0f81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0f7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 0x7fb0f81974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45088/0 (socket says 172.21.15.82:45088) 2023-07-14T19:39:36.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0f7fff700 1 -- 172.21.15.82:0/4139061320 learned_addr learned my addr 172.21.15.82:0/4139061320 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:36.107 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.084+0000 7fb0f77fe700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0f80fe260 0x7fb0f8106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.085+0000 7fb0f7fff700 1 -- 172.21.15.82:0/4139061320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 msgr2=0x7fb0f8106560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.085+0000 7fb0f7fff700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 0x7fb0f8106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.085+0000 7fb0f7fff700 1 -- 172.21.15.82:0/4139061320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0f80fe260 msgr2=0x7fb0f8106020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.085+0000 7fb0f7fff700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0f80fe260 0x7fb0f8106020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.085+0000 7fb0f7fff700 1 -- 172.21.15.82:0/4139061320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec01a600 con 0x7fb0f8103110 2023-07-14T19:39:36.108 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.085+0000 7fb0f7fff700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 0x7fb0f81974c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000b970 tx=0x7fb0f000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.091+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000c9b0 con 0x7fb0f8103110 2023-07-14T19:39:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.091+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f0014440 con 0x7fb0f8103110 2023-07-14T19:39:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.091+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0007550 con 0x7fb0f8103110 2023-07-14T19:39:36.121 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.092+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f8197ea0 con 0x7fb0f8103110 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.092+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f8198330 con 0x7fb0f8103110 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.097+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb0f00145b0 con 0x7fb0f8103110 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.098+0000 7fb0f4ff9700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb0e00a1520 0x7fb0e00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.098+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f8067880 con 0x7fb0f8103110 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.098+0000 7fb0f77fe700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb0e00a1520 0x7fb0e00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.098+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fb0f00cbf50 con 0x7fb0f8103110 2023-07-14T19:39:36.122 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.098+0000 7fb0f77fe700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb0e00a1520 0x7fb0e00a39d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009910 tx=0x7fb0e8008040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.106+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fb0f004d780 con 0x7fb0f8103110 2023-07-14T19:39:36.238 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:35 smithi082 ceph-mon[127461]: pgmap v345: 97 pgs: 39 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 37 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-14T19:39:36.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.438+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0f8198ad0 con 0x7fb0f8103110 2023-07-14T19:39:36.462 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.440+0000 7fb0f4ff9700 1 -- 172.21.15.82:0/4139061320 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+22900 (secure 0 0 0) 0x7fb0f0019080 con 0x7fb0f8103110 2023-07-14T19:39:36.462 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:39:36.463 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":96,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:39:11.569714+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6801/504215117":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/3311699035":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/1722617918":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/3623265050":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1617843074":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/504215117":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:39:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.444+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb0e00a1520 msgr2=0x7fb0e00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.444+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb0e00a1520 0x7fb0e00a39d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009910 tx=0x7fb0e8008040 comp rx=0 tx=0).stop 2023-07-14T19:39:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.444+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 msgr2=0x7fb0f81974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.444+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 0x7fb0f81974c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000b970 tx=0x7fb0f000bd30 comp rx=0 tx=0).stop 2023-07-14T19:39:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.445+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 shutdown_connections 2023-07-14T19:39:36.466 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.445+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fb0e00a1520 0x7fb0e00a39d0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009910 tx=0x7fb0e8008040 comp rx=0 tx=0).stop 2023-07-14T19:39:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.445+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0f8103110 0x7fb0f81974c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.445+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0f80fe260 0x7fb0f8106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.445+0000 7fb0fdf64700 1 --2- 172.21.15.82:0/4139061320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0f80fec00 0x7fb0f8106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.445+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 >> 172.21.15.82:0/4139061320 conn(0x7fb0f8074db0 msgr2=0x7fb0f8101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:36.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.449+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 shutdown_connections 2023-07-14T19:39:36.472 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.451+0000 7fb0fdf64700 1 -- 172.21.15.82:0/4139061320 wait complete. 2023-07-14T19:39:36.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-14T19:23:08.979526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-14T19:26:54.722744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-14T19:26:56.092558+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-14T19:39:36.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-14T19:39:36.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.707+0000 7efc0770f700 1 -- 172.21.15.82:0/694847482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00102e80 msgr2=0x7efc001032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.729 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.707+0000 7efc0770f700 1 --2- 172.21.15.82:0/694847482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00102e80 0x7efc001032d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009f90 tx=0x7efbf800b330 comp rx=0 tx=0).stop 2023-07-14T19:39:36.730 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 -- 172.21.15.82:0/694847482 shutdown_connections 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 --2- 172.21.15.82:0/694847482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00102e80 0x7efc001032d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 --2- 172.21.15.82:0/694847482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc00066f10 0x7efc000fe5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 --2- 172.21.15.82:0/694847482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00067370 0x7efc000feae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 -- 172.21.15.82:0/694847482 >> 172.21.15.82:0/694847482 conn(0x7efc00076330 msgr2=0x7efc00076730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 -- 172.21.15.82:0/694847482 shutdown_connections 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.708+0000 7efc0770f700 1 -- 172.21.15.82:0/694847482 wait complete. 2023-07-14T19:39:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 Processor -- start 2023-07-14T19:39:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 -- start start 2023-07-14T19:39:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 0x7efc0010ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00067370 0x7efc0010f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc00102e80 0x7efc00111840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00111e10 con 0x7efc00066f10 2023-07-14T19:39:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00111f80 con 0x7efc00102e80 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc0770f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00112140 con 0x7efc00067370 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc054ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 0x7efc0010ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc054ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 0x7efc0010ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45100/0 (socket says 172.21.15.82:45100) 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc054ab700 1 -- 172.21.15.82:0/3463970251 learned_addr learned my addr 172.21.15.82:0/3463970251 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc04caa700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00067370 0x7efc0010f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.709+0000 7efc05cac700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc00102e80 0x7efc00111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efc054ab700 1 -- 172.21.15.82:0/3463970251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00067370 msgr2=0x7efc0010f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efc054ab700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00067370 0x7efc0010f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efc054ab700 1 -- 172.21.15.82:0/3463970251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc00102e80 msgr2=0x7efc00111840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efc054ab700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc00102e80 0x7efc00111840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efc054ab700 1 -- 172.21.15.82:0/3463970251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00095f0 con 0x7efc00066f10 2023-07-14T19:39:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efc054ab700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 0x7efc0010ee30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efbf000ec10 tx=0x7efbf000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.710+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf000cc90 con 0x7efc00066f10 2023-07-14T19:39:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.711+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbf00044e0 con 0x7efc00066f10 2023-07-14T19:39:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.711+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0010430 con 0x7efc00066f10 2023-07-14T19:39:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.711+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8009c40 con 0x7efc00066f10 2023-07-14T19:39:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.711+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0019e840 con 0x7efc00066f10 2023-07-14T19:39:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.714+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efbf0003f90 con 0x7efc00066f10 2023-07-14T19:39:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.715+0000 7efbf67fc700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efbec0a1410 0x7efbec0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.715+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc00066b30 con 0x7efc00066f10 2023-07-14T19:39:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.715+0000 7efc04caa700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efbec0a1410 0x7efbec0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.715+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7efbf0014070 con 0x7efc00066f10 2023-07-14T19:39:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.716+0000 7efc04caa700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efbec0a1410 0x7efbec0a38c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc00110440 tx=0x7efbfc009190 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:36.744 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:36.722+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7efbf0096550 con 0x7efc00066f10 2023-07-14T19:39:37.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.061+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc0019eb20 con 0x7efc00066f10 2023-07-14T19:39:37.085 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.062+0000 7efbf67fc700 1 -- 172.21.15.82:0/3463970251 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7efbf006c5b0 con 0x7efc00066f10 2023-07-14T19:39:37.086 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2023-07-14T19:39:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.067+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efbec0a1410 msgr2=0x7efbec0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.067+0000 7efc0770f700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efbec0a1410 0x7efbec0a38c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc00110440 tx=0x7efbfc009190 comp rx=0 tx=0).stop 2023-07-14T19:39:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.067+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 msgr2=0x7efc0010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.067+0000 7efc0770f700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 0x7efc0010ee30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efbf000ec10 tx=0x7efbf000ef20 comp rx=0 tx=0).stop 2023-07-14T19:39:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.068+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 shutdown_connections 2023-07-14T19:39:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.068+0000 7efc0770f700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7efbec0a1410 0x7efbec0a38c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.068+0000 7efc0770f700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc00102e80 0x7efc00111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.068+0000 7efc0770f700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc00066f10 0x7efc0010ee30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.068+0000 7efc0770f700 1 --2- 172.21.15.82:0/3463970251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc00067370 0x7efc0010f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.091 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.068+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 >> 172.21.15.82:0/3463970251 conn(0x7efc00076330 msgr2=0x7efc000fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:37.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.070+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 shutdown_connections 2023-07-14T19:39:37.092 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.070+0000 7efc0770f700 1 -- 172.21.15.82:0/3463970251 wait complete. 2023-07-14T19:39:37.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-14T19:39:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 -- 172.21.15.82:0/1639776424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 msgr2=0x7f9ea010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/1639776424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 0x7f9ea010b920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9e900098f0 tx=0x7f9e90009c00 comp rx=0 tx=0).stop 2023-07-14T19:39:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 -- 172.21.15.82:0/1639776424 shutdown_connections 2023-07-14T19:39:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/1639776424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 0x7f9ea010b920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/1639776424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea00ff840 0x7f9ea00ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/1639776424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea01001e0 0x7f9ea01036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 -- 172.21.15.82:0/1639776424 >> 172.21.15.82:0/1639776424 conn(0x7f9ea0075c30 msgr2=0x7f9ea0076030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.317+0000 7f9ea64fe700 1 -- 172.21.15.82:0/1639776424 shutdown_connections 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.318+0000 7f9ea64fe700 1 -- 172.21.15.82:0/1639776424 wait complete. 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.318+0000 7f9ea64fe700 1 Processor -- start 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.318+0000 7f9ea64fe700 1 -- start start 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea64fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 0x7f9ea019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.342 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea64fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea01001e0 0x7f9ea019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea64fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 0x7f9ea0195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea64fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0196150 con 0x7f9ea00ff840 2023-07-14T19:39:37.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea64fe700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01962c0 con 0x7f9ea0103d90 2023-07-14T19:39:37.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea64fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0196480 con 0x7f9ea01001e0 2023-07-14T19:39:37.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 0x7f9ea019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:37.343 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 0x7f9ea019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45114/0 (socket says 172.21.15.82:45114) 2023-07-14T19:39:37.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 -- 172.21.15.82:0/3496717985 learned_addr learned my addr 172.21.15.82:0/3496717985 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:37.344 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9ea4a9b700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 0x7f9ea0195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:37.345 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 -- 172.21.15.82:0/3496717985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea01001e0 msgr2=0x7f9ea019bfb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:39:37.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea01001e0 0x7f9ea019bfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 -- 172.21.15.82:0/3496717985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 msgr2=0x7f9ea0195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 0x7f9ea0195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.319+0000 7f9e9ffff700 1 -- 172.21.15.82:0/3496717985 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90009520 con 0x7f9ea00ff840 2023-07-14T19:39:37.346 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.320+0000 7f9e9ffff700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 0x7f9ea019ba70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9e980060b0 tx=0x7f9e9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.325+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800ddd0 con 0x7f9ea00ff840 2023-07-14T19:39:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.326+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e9800f040 con 0x7f9ea00ff840 2023-07-14T19:39:37.347 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.326+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980146d0 con 0x7f9ea00ff840 2023-07-14T19:39:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.327+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea0196760 con 0x7f9ea00ff840 2023-07-14T19:39:37.348 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.327+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea01a2a60 con 0x7f9ea00ff840 2023-07-14T19:39:37.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.332+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9e98004ad0 con 0x7f9ea00ff840 2023-07-14T19:39:37.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.332+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea00677a0 con 0x7f9ea00ff840 2023-07-14T19:39:37.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.332+0000 7f9e9d7fa700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9e880a1490 0x7f9e880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.357 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.333+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f9e980cc310 con 0x7f9ea00ff840 2023-07-14T19:39:37.358 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.337+0000 7f9e9f7fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9e880a1490 0x7f9e880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:37.359 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.338+0000 7f9e9f7fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9e880a1490 0x7f9e880a3940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9ea01051e0 tx=0x7f9e94009310 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:37.361 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.339+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f9e980956f0 con 0x7f9ea00ff840 2023-07-14T19:39:37.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:37 smithi133 ceph-mon[128894]: pgmap v346: 97 pgs: 33 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2023-07-14T19:39:37.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:37 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4139061320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:39:37.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:37 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3463970251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:39:37.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:37 smithi082 ceph-mon[127461]: pgmap v346: 97 pgs: 33 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2023-07-14T19:39:37.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:37 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4139061320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:39:37.485 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:37 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3463970251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:39:37.514 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:37 smithi161 ceph-mon[129184]: pgmap v346: 97 pgs: 33 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 48 active+clean; 1.8 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2023-07-14T19:39:37.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:37 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4139061320' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:39:37.515 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:37 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3463970251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-14T19:39:37.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.669+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ea0196ff0 con 0x7f9ea00ff840 2023-07-14T19:39:37.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.672+0000 7f9e9d7fa700 1 -- 172.21.15.82:0/3496717985 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ea0196ff0 con 0x7f9ea00ff840 2023-07-14T19:39:37.694 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2023-07-14T19:39:37.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9e880a1490 msgr2=0x7f9e880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9e880a1490 0x7f9e880a3940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9ea01051e0 tx=0x7f9e94009310 comp rx=0 tx=0).stop 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 msgr2=0x7f9ea019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 0x7f9ea019ba70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9e980060b0 tx=0x7f9e9800d3b0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 shutdown_connections 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f9e880a1490 0x7f9e880a3940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea0103d90 0x7f9ea0195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea00ff840 0x7f9ea019ba70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 --2- 172.21.15.82:0/3496717985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea01001e0 0x7f9ea019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 >> 172.21.15.82:0/3496717985 conn(0x7f9ea0075c30 msgr2=0x7f9ea00fb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:37.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 shutdown_connections 2023-07-14T19:39:37.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.680+0000 7f9ea64fe700 1 -- 172.21.15.82:0/3496717985 wait complete. 2023-07-14T19:39:37.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-14T19:39:37.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.926+0000 7f3c164c5700 1 -- 172.21.15.82:0/341522170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c100fdd20 msgr2=0x7f3c100fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.926+0000 7f3c164c5700 1 --2- 172.21.15.82:0/341522170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c100fdd20 0x7f3c100fe190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009990 tx=0x7f3c00009ca0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 -- 172.21.15.82:0/341522170 shutdown_connections 2023-07-14T19:39:37.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 --2- 172.21.15.82:0/341522170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c100fe6d0 0x7f3c1010ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 --2- 172.21.15.82:0/341522170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c10105cf0 0x7f3c101060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.949 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 --2- 172.21.15.82:0/341522170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c100fdd20 0x7f3c100fe190 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 -- 172.21.15.82:0/341522170 >> 172.21.15.82:0/341522170 conn(0x7f3c10075ca0 msgr2=0x7f3c100760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 -- 172.21.15.82:0/341522170 shutdown_connections 2023-07-14T19:39:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 -- 172.21.15.82:0/341522170 wait complete. 2023-07-14T19:39:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 Processor -- start 2023-07-14T19:39:37.951 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.927+0000 7f3c164c5700 1 -- start start 2023-07-14T19:39:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c164c5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c100fdd20 0x7f3c100ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c164c5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c100fe6d0 0x7f3c100ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c164c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 0x7f3c101a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c164c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10100500 con 0x7f3c10105cf0 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c164c5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101a4410 con 0x7f3c100fe6d0 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c164c5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101a4550 con 0x7f3c100fdd20 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c14a62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 0x7f3c101a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c14a62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 0x7f3c101a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45128/0 (socket says 172.21.15.82:45128) 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c14a62700 1 -- 172.21.15.82:0/1120063704 learned_addr learned my addr 172.21.15.82:0/1120063704 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.928+0000 7f3c0f7fe700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c100fe6d0 0x7f3c100ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c14a62700 1 -- 172.21.15.82:0/1120063704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c100fdd20 msgr2=0x7f3c100ff790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c14a62700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c100fdd20 0x7f3c100ff790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c14a62700 1 -- 172.21.15.82:0/1120063704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c100fe6d0 msgr2=0x7f3c100ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c14a62700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c100fe6d0 0x7f3c100ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c14a62700 1 -- 172.21.15.82:0/1120063704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c000095f0 con 0x7f3c10105cf0 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c14a62700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 0x7f3c101a3ed0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800efc0 tx=0x7f3c0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08018070 con 0x7f3c10105cf0 2023-07-14T19:39:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c0800f040 con 0x7f3c10105cf0 2023-07-14T19:39:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.929+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800cc90 con 0x7f3c10105cf0 2023-07-14T19:39:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.930+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c10195b10 con 0x7f3c10105cf0 2023-07-14T19:39:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.930+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c10196020 con 0x7f3c10105cf0 2023-07-14T19:39:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.932+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c10067810 con 0x7f3c10105cf0 2023-07-14T19:39:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.934+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3c08004ad0 con 0x7f3c10105cf0 2023-07-14T19:39:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.934+0000 7f3c0d7fa700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3bf80a1340 0x7f3bf80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.934+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f3c080cc790 con 0x7f3c10105cf0 2023-07-14T19:39:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.935+0000 7f3c0ffff700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3bf80a1340 0x7f3bf80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.935+0000 7f3c0ffff700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3bf80a1340 0x7f3bf80a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009730 tx=0x7f3c04006cb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:37.941+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f3c08095b60 con 0x7f3c10105cf0 2023-07-14T19:39:38.289 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.267+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3c101963a0 con 0x7f3c10105cf0 2023-07-14T19:39:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.270+0000 7f3c0d7fa700 1 -- 172.21.15.82:0/1120063704 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3c08099180 con 0x7f3c10105cf0 2023-07-14T19:39:38.295 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2023-07-14T19:39:38.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3bf80a1340 msgr2=0x7f3bf80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3bf80a1340 0x7f3bf80a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009730 tx=0x7f3c04006cb0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 msgr2=0x7f3c101a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 0x7f3c101a3ed0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800efc0 tx=0x7f3c0800c3a0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 shutdown_connections 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f3bf80a1340 0x7f3bf80a37f0 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009730 tx=0x7f3c04006cb0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10105cf0 0x7f3c101a3ed0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c100fdd20 0x7f3c100ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 --2- 172.21.15.82:0/1120063704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c100fe6d0 0x7f3c100ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.276+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 >> 172.21.15.82:0/1120063704 conn(0x7f3c10075ca0 msgr2=0x7f3c100fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:38.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.280+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 shutdown_connections 2023-07-14T19:39:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.281+0000 7f3c164c5700 1 -- 172.21.15.82:0/1120063704 wait complete. 2023-07-14T19:39:38.323 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-14T19:39:38.457 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:38 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3496717985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:39:38.484 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:38 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3496717985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:39:38.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:38 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3496717985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-14T19:39:38.532 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.507+0000 7fc84dcc9700 1 -- 172.21.15.82:0/103767843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 msgr2=0x7fc848104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.533 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.507+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/103767843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc848104ee0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc838009990 tx=0x7fc838009ca0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 -- 172.21.15.82:0/103767843 shutdown_connections 2023-07-14T19:39:38.534 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/103767843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480ff4b0 0x7fc84810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/103767843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc848104ee0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/103767843 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480feb00 0x7fc8480fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 -- 172.21.15.82:0/103767843 >> 172.21.15.82:0/103767843 conn(0x7fc8480fa5e0 msgr2=0x7fc8480fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:38.535 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 -- 172.21.15.82:0/103767843 shutdown_connections 2023-07-14T19:39:38.536 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.512+0000 7fc84dcc9700 1 -- 172.21.15.82:0/103767843 wait complete. 2023-07-14T19:39:38.538 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 Processor -- start 2023-07-14T19:39:38.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 -- start start 2023-07-14T19:39:38.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 0x7fc84806ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:38.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480ff4b0 0x7fc84806b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:38.539 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc8481a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8481a8670 con 0x7fc848104b10 2023-07-14T19:39:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8481a87e0 con 0x7fc8480feb00 2023-07-14T19:39:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc84dcc9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8481a89a0 con 0x7fc8480ff4b0 2023-07-14T19:39:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc8477fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 0x7fc84806ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc847fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc8481a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc8477fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 0x7fc84806ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41488/0 (socket says 172.21.15.82:41488) 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc8477fe700 1 -- 172.21.15.82:0/1429090277 learned_addr learned my addr 172.21.15.82:0/1429090277 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.513+0000 7fc83ffff700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480ff4b0 0x7fc84806b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc847fff700 1 -- 172.21.15.82:0/1429090277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480ff4b0 msgr2=0x7fc84806b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc847fff700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480ff4b0 0x7fc84806b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc847fff700 1 -- 172.21.15.82:0/1429090277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 msgr2=0x7fc84806ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc847fff700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 0x7fc84806ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc847fff700 1 -- 172.21.15.82:0/1429090277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380095f0 con 0x7fc848104b10 2023-07-14T19:39:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc83ffff700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480ff4b0 0x7fc84806b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:39:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc8477fe700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 0x7fc84806ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:39:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.514+0000 7fc847fff700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc8481a80a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc840007f10 tx=0x7fc84000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.516+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840009ea0 con 0x7fc848104b10 2023-07-14T19:39:38.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.516+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84000f040 con 0x7fc848104b10 2023-07-14T19:39:38.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.516+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840004350 con 0x7fc848104b10 2023-07-14T19:39:38.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.518+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8481a8c80 con 0x7fc848104b10 2023-07-14T19:39:38.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.519+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8481a9140 con 0x7fc848104b10 2023-07-14T19:39:38.543 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.520+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8480673c0 con 0x7fc848104b10 2023-07-14T19:39:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.521+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc8400044b0 con 0x7fc848104b10 2023-07-14T19:39:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.522+0000 7fc8457fa700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc8340a1410 0x7fc8340a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.522+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7fc8400cc4a0 con 0x7fc848104b10 2023-07-14T19:39:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.524+0000 7fc8477fe700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc8340a1410 0x7fc8340a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.524+0000 7fc8477fe700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc8340a1410 0x7fc8340a38c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838006010 tx=0x7fc838005ab0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:38.559 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.531+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7fc840095d60 con 0x7fc848104b10 2023-07-14T19:39:38.902 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.880+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc84806bf80 con 0x7fc848104b10 2023-07-14T19:39:38.907 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.885+0000 7fc8457fa700 1 -- 172.21.15.82:0/1429090277 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) v1 ==== 81+0+940 (secure 0 0 0) 0x7fc840098f20 con 0x7fc848104b10 2023-07-14T19:39:38.910 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/504215117 2023-07-15T19:28:33.481511+0000 2023-07-14T19:39:38.910 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3311699035 2023-07-15T19:28:33.481511+0000 2023-07-14T19:39:38.910 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1722617918 2023-07-15T19:28:33.481511+0000 2023-07-14T19:39:38.910 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3623265050 2023-07-15T19:28:33.481511+0000 2023-07-14T19:39:38.910 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/2410218140 2023-07-15T19:21:04.644444+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1617843074 2023-07-15T19:28:33.481511+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/1257676906 2023-07-15T19:21:04.644444+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/1035467551 2023-07-15T19:21:04.644444+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/1035467551 2023-07-15T19:21:04.644444+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6801/3383156270 2023-07-15T19:20:42.182360+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/504215117 2023-07-15T19:28:33.481511+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/998146971 2023-07-15T19:20:42.182360+0000 2023-07-14T19:39:38.911 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6800/3383156270 2023-07-15T19:20:42.182360+0000 2023-07-14T19:39:38.912 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/690674119 2023-07-15T19:20:42.182360+0000 2023-07-14T19:39:38.912 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/3474653214 2023-07-15T19:21:04.644444+0000 2023-07-14T19:39:38.912 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:0/10878106 2023-07-15T19:20:42.182360+0000 2023-07-14T19:39:38.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc8340a1410 msgr2=0x7fc8340a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc8340a1410 0x7fc8340a38c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838006010 tx=0x7fc838005ab0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 msgr2=0x7fc8481a80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:38.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc8481a80a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc840007f10 tx=0x7fc84000d3b0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 shutdown_connections 2023-07-14T19:39:38.913 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7fc8340a1410 0x7fc8340a38c0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc838006010 tx=0x7fc838005ab0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc848104b10 0x7fc8481a80a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8480feb00 0x7fc84806ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 --2- 172.21.15.82:0/1429090277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8480ff4b0 0x7fc84806b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:38.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.891+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 >> 172.21.15.82:0/1429090277 conn(0x7fc8480fa5e0 msgr2=0x7fc8480fabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:38.914 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.892+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 shutdown_connections 2023-07-14T19:39:38.915 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:38.892+0000 7fc84dcc9700 1 -- 172.21.15.82:0/1429090277 wait complete. 2023-07-14T19:39:38.932 INFO:teuthology.orchestra.run.smithi082.stderr:listed 16 entries 2023-07-14T19:39:38.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-14T19:39:38.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:38.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-14T19:39:39.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:39 smithi082 ceph-mon[127461]: pgmap v347: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T19:39:39.236 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:39 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1120063704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:39:39.237 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:39 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1429090277' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T19:39:39.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:39:39.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:39.248 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-14T19:39:39.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:39.430 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2023-07-14T19:39:39.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:39 smithi161 ceph-mon[129184]: pgmap v347: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T19:39:39.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:39 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1120063704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:39:39.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:39 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1429090277' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T19:39:39.567 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-14T19:39:39.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:39.568 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-14T19:39:39.711 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:39 smithi133 ceph-mon[128894]: pgmap v347: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-14T19:39:39.712 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:39 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1120063704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-14T19:39:39.712 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:39 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1429090277' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-14T19:39:39.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:39.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2023-07-14T19:39:39.885 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-14T19:39:39.886 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-07-14T19:39:39.886 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-14T19:39:39.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17166 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/60db04c00d7d6d7a5744ee06adef2f4ab52f5fe6ca931ead4e35290e74b9adbc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/60db04c00d7d6d7a5744ee06adef2f4ab52f5fe6ca931ead4e35290e74b9adbc/diff,workdir=/var/lib/containers/storage/overlay/60db04c00d7d6d7a5744ee06adef2f4ab52f5fe6ca931ead4e35290e74b9adbc/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9c06b2dfc24b98426f5868da2921593b41db284daa31ee5036339696071f1a31/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/9c06b2dfc24b98426f5868da2921593b41db284daa31ee5036339696071f1a31/diff,workdir=/var/lib/containers/storage/overlay/9c06b2dfc24b98426f5868da2921593b41db284daa31ee5036339696071f1a31/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/151da8da634236835f46367ece6528800cb32d2e17bfdc2e72c92c8631174f67/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/151da8da634236835f46367ece6528800cb32d2e17bfdc2e72c92c8631174f67/diff,workdir=/var/lib/containers/storage/overlay/151da8da634236835f46367ece6528800cb32d2e17bfdc2e72c92c8631174f67/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ded2fdcd6acc14b9f65fac5f1daa29a881c6efa6d416d6fb6663ed112a752d7a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/ded2fdcd6acc14b9f65fac5f1daa29a881c6efa6d416d6fb6663ed112a752d7a/diff,workdir=/var/lib/containers/storage/overlay/ded2fdcd6acc14b9f65fac5f1daa29a881c6efa6d416d6fb6663ed112a752d7a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e13d8b0824df5470d5c4c91000c4fc820a323ffdd024ab9897db739d281dd3b1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/e13d8b0824df5470d5c4c91000c4fc820a323ffdd024ab9897db739d281dd3b1/diff,workdir=/var/lib/containers/storage/overlay/e13d8b0824df5470d5c4c91000c4fc820a323ffdd024ab9897db739d281dd3b1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/55c6b46209d3fe73bc9439df25e547ba713c79e54236af389876e06b18dfb49b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/55c6b46209d3fe73bc9439df25e547ba713c79e54236af389876e06b18dfb49b/diff,workdir=/var/lib/containers/storage/overlay/55c6b46209d3fe73bc9439df25e547ba713c79e54236af389876e06b18dfb49b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d50a912d259b4e7d4f82661242fde0fcc721d287b17022075c33b6b428d81148/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/d50a912d259b4e7d4f82661242fde0fcc721d287b17022075c33b6b428d81148/diff,workdir=/var/lib/containers/storage/overlay/d50a912d259b4e7d4f82661242fde0fcc721d287b17022075c33b6b428d81148/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a8074677c489e25bde5ef81634bea70c46845e6bbfc642134114cf18c149e123/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/a8074677c489e25bde5ef81634bea70c46845e6bbfc642134114cf18c149e123/diff,workdir=/var/lib/containers/storage/overlay/a8074677c489e25bde5ef81634bea70c46845e6bbfc642134114cf18c149e123/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9ece4f6ab054e342be5dd22c613e777fe04a4b9444ca117a5dc30c929f82073a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/9ece4f6ab054e342be5dd22c613e777fe04a4b9444ca117a5dc30c929f82073a/diff,workdir=/var/lib/containers/storage/overlay/9ece4f6ab054e342be5dd22c613e777fe04a4b9444ca117a5dc30c929f82073a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b361b3583c61d5eff7a8afd54f15fd2df23af5b181bbda4d0e655e0a145a22c0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/KPZAPXUISI5GF7NFX6DZMI2YEO:/var/lib/containers/storage/overlay/l/ETSJAE7STELGKOWCE5CYLBZQAV,upperdir=/var/lib/containers/storage/overlay/b361b3583c61d5eff7a8afd54f15fd2df23af5b181bbda4d0e655e0a145a22c0/diff,workdir=/var/lib/containers/storage/overlay/b361b3583c61d5eff7a8afd54f15fd2df23af5b181bbda4d0e655e0a145a22c0/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-14T19:39:39.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:39.916 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2023-07-14T19:39:39.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-14T19:39:39.989 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2023-07-14T19:39:40.089 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2023-07-14T19:39:40.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:39:40.094 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-14T19:39:40.106 INFO:tasks.cephadm:Teardown begin 2023-07-14T19:39:40.107 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-14T19:39:40.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:39:40.295 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 -- 172.21.15.82:0/91754687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006be30 msgr2=0x7f5ef006c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 --2- 172.21.15.82:0/91754687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006be30 0x7f5ef006c200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00099c0 tx=0x7f5ee0009cd0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 -- 172.21.15.82:0/91754687 shutdown_connections 2023-07-14T19:39:40.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 --2- 172.21.15.82:0/91754687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef010c630 0x7f5ef01141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 --2- 172.21.15.82:0/91754687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006be30 0x7f5ef006c200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.296 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 --2- 172.21.15.82:0/91754687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006c7d0 0x7f5ef010bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.273+0000 7f5ef829d700 1 -- 172.21.15.82:0/91754687 >> 172.21.15.82:0/91754687 conn(0x7f5ef006b1c0 msgr2=0x7f5ef0071050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:40.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.274+0000 7f5ef829d700 1 -- 172.21.15.82:0/91754687 shutdown_connections 2023-07-14T19:39:40.297 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.274+0000 7f5ef829d700 1 -- 172.21.15.82:0/91754687 wait complete. 2023-07-14T19:39:40.298 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.274+0000 7f5ef829d700 1 Processor -- start 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.274+0000 7f5ef829d700 1 -- start start 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef829d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 0x7f5ef011c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef829d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006c7d0 0x7f5ef011c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef829d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef010c630 0x7f5ef0117510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef829d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0117ce0 con 0x7f5ef006be30 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef829d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0117e50 con 0x7f5ef010c630 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef829d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0118010 con 0x7f5ef006c7d0 2023-07-14T19:39:40.299 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 0x7f5ef011c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 0x7f5ef011c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45154/0 (socket says 172.21.15.82:45154) 2023-07-14T19:39:40.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 -- 172.21.15.82:0/644046443 learned_addr learned my addr 172.21.15.82:0/644046443 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:40.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef5838700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006c7d0 0x7f5ef011c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef683a700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef010c630 0x7f5ef0117510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 -- 172.21.15.82:0/644046443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006c7d0 msgr2=0x7f5ef011c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.300 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006c7d0 0x7f5ef011c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 -- 172.21.15.82:0/644046443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef010c630 msgr2=0x7f5ef0117510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef010c630 0x7f5ef0117510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 -- 172.21.15.82:0/644046443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec009520 con 0x7f5ef006be30 2023-07-14T19:39:40.301 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef5838700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006c7d0 0x7f5ef011c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-14T19:39:40.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.275+0000 7f5ef6039700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 0x7f5ef011c450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00036f0 tx=0x7f5ee000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:40.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.276+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee001d070 con 0x7f5ef006be30 2023-07-14T19:39:40.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.276+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee0005050 con 0x7f5ef006be30 2023-07-14T19:39:40.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.276+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004ab0 con 0x7f5ef006be30 2023-07-14T19:39:40.303 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.276+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee00095f0 con 0x7f5ef006be30 2023-07-14T19:39:40.304 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.276+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef01201c0 con 0x7f5ef006be30 2023-07-14T19:39:40.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.277+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0067430 con 0x7f5ef006be30 2023-07-14T19:39:40.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.279+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5ee0004c70 con 0x7f5ef006be30 2023-07-14T19:39:40.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.279+0000 7f5ee77fe700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5edc0a1370 0x7f5edc0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.280+0000 7f5ef5838700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5edc0a1370 0x7f5edc0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.280+0000 7f5ef5838700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5edc0a1370 0x7f5edc0a3820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0118ce0 tx=0x7f5eec009260 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:40.308 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.286+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f5ee009ad00 con 0x7f5ef006be30 2023-07-14T19:39:40.309 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.287+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f5ee0097040 con 0x7f5ef006be30 2023-07-14T19:39:40.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.671+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ef0118a20 con 0x7f5ef006be30 2023-07-14T19:39:40.694 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.672+0000 7f5ee77fe700 1 -- 172.21.15.82:0/644046443 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+10839 (secure 0 0 0) 0x7f5ee009a660 con 0x7f5ef006be30 2023-07-14T19:39:40.694 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:39:40.695 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:39:10.564475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":37,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25678":{"gid":25678,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3620688590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3620688590},{"type":"v1","addr":"172.21.15.161:6837","nonce":3620688590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:39:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5edc0a1370 msgr2=0x7f5edc0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5edc0a1370 0x7f5edc0a3820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0118ce0 tx=0x7f5eec009260 comp rx=0 tx=0).stop 2023-07-14T19:39:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 msgr2=0x7f5ef011c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 0x7f5ef011c450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00036f0 tx=0x7f5ee000f690 comp rx=0 tx=0).stop 2023-07-14T19:39:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 shutdown_connections 2023-07-14T19:39:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f5edc0a1370 0x7f5edc0a3820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ef010c630 0x7f5ef0117510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef006be30 0x7f5ef011c450 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.679+0000 7f5ef829d700 1 --2- 172.21.15.82:0/644046443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ef006c7d0 0x7f5ef011c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.680+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 >> 172.21.15.82:0/644046443 conn(0x7f5ef006b1c0 msgr2=0x7f5ef010afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.680+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 shutdown_connections 2023-07-14T19:39:40.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.680+0000 7f5ef829d700 1 -- 172.21.15.82:0/644046443 wait complete. 2023-07-14T19:39:40.705 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 47 2023-07-14T19:39:40.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-14T19:39:40.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 -- 172.21.15.82:0/1124238421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806bbc0 msgr2=0x7f2db806c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1124238421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806bbc0 0x7f2db806c010 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009990 tx=0x7f2da8009ca0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 -- 172.21.15.82:0/1124238421 shutdown_connections 2023-07-14T19:39:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1124238421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2db806c630 0x7f2db811b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1124238421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db810e330 0x7f2db810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1124238421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806bbc0 0x7f2db806c010 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 -- 172.21.15.82:0/1124238421 >> 172.21.15.82:0/1124238421 conn(0x7f2db806b1c0 msgr2=0x7f2db8071050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 -- 172.21.15.82:0/1124238421 shutdown_connections 2023-07-14T19:39:40.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.895+0000 7f2dc0711700 1 -- 172.21.15.82:0/1124238421 wait complete. 2023-07-14T19:39:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 Processor -- start 2023-07-14T19:39:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 -- start start 2023-07-14T19:39:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 0x7f2db81aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806c630 0x7f2db81aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2db810e330 0x7f2db81a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.920 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81a5150 con 0x7f2db806c630 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81a52c0 con 0x7f2db806bbc0 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dc0711700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db81a5480 con 0x7f2db810e330 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dbdcac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806c630 0x7f2db81aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dbe4ad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 0x7f2db81aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dbe4ad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 0x7f2db81aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41516/0 (socket says 172.21.15.82:41516) 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dbdcac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806c630 0x7f2db81aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45172/0 (socket says 172.21.15.82:45172) 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dbe4ad700 1 -- 172.21.15.82:0/1648595688 learned_addr learned my addr 172.21.15.82:0/1648595688 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:40.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.896+0000 7f2dbecae700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2db810e330 0x7f2db81a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dbe4ad700 1 -- 172.21.15.82:0/1648595688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2db810e330 msgr2=0x7f2db81a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dbe4ad700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2db810e330 0x7f2db81a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dbe4ad700 1 -- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806c630 msgr2=0x7f2db81aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dbe4ad700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806c630 0x7f2db81aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dbe4ad700 1 -- 172.21.15.82:0/1648595688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80095f0 con 0x7f2db806bbc0 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dbe4ad700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 0x7f2db81aaa70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2db000b840 tx=0x7f2db000bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:40.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0010800 con 0x7f2db806bbc0 2023-07-14T19:39:40.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db81a5760 con 0x7f2db806bbc0 2023-07-14T19:39:40.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db0010e20 con 0x7f2db806bbc0 2023-07-14T19:39:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0017360 con 0x7f2db806bbc0 2023-07-14T19:39:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.897+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db81b1a60 con 0x7f2db806bbc0 2023-07-14T19:39:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.898+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db8067430 con 0x7f2db806bbc0 2023-07-14T19:39:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.900+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2db0007590 con 0x7f2db806bbc0 2023-07-14T19:39:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.901+0000 7f2daf7fe700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2da40a1380 0x7f2da40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:40.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.901+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f2db00cc8f0 con 0x7f2db806bbc0 2023-07-14T19:39:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.901+0000 7f2dbdcac700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2da40a1380 0x7f2da40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:40.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.902+0000 7f2dbdcac700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2da40a1380 0x7f2da40a3830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2db810cc70 tx=0x7f2da800b540 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:40.929 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:40.907+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f2db0095d30 con 0x7f2db806bbc0 2023-07-14T19:39:41.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.284+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2db81a5ff0 con 0x7f2db806bbc0 2023-07-14T19:39:41.311 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.289+0000 7f2daf7fe700 1 -- 172.21.15.82:0/1648595688 <== mon.2 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+10837 (secure 0 0 0) 0x7f2db0099350 con 0x7f2db806bbc0 2023-07-14T19:39:41.312 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:39:41.312 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1071311199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1071311199},{"type":"v1","addr":"172.21.15.82:6835","nonce":1071311199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/464864339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":464864339},{"type":"v1","addr":"172.21.15.82:6837","nonce":464864339}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14622,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/3224571334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":3224571334},{"type":"v1","addr":"172.21.15.82:6839","nonce":3224571334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6841/2495420363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6840","nonce":2495420363},{"type":"v1","addr":"172.21.15.82:6841","nonce":2495420363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/2795796645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":2795796645},{"type":"v1","addr":"172.21.15.161:6833","nonce":2795796645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1264023541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1264023541},{"type":"v1","addr":"172.21.15.133:6833","nonce":1264023541}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":178,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"balance_automate":true},"ever_allowed_features":160,"explicitly_allowed_features":160,"created":"2023-07-14T19:26:56.908603+0000","modified":"2023-07-14T19:39:41.159985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24491,"mds_2":24481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6835/3902735752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3902735752},{"type":"v1","addr":"172.21.15.161:6835","nonce":3902735752}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/3278917095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3278917095},{"type":"v1","addr":"172.21.15.133:6835","nonce":3278917095}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24481":{"gid":24481,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.161:6839/4007435956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":4007435956},{"type":"v1","addr":"172.21.15.161:6839","nonce":4007435956}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24491":{"gid":24491,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":37,"addr":"172.21.15.133:6837/46070579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":46070579},{"type":"v1","addr":"172.21.15.133:6837","nonce":46070579}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6839/242970631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":242970631},{"type":"v1","addr":"172.21.15.133:6839","nonce":242970631}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25678":{"gid":25678,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6837/3620688590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":3620688590},{"type":"v1","addr":"172.21.15.161:6837","nonce":3620688590}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-14T19:39:41.314 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2da40a1380 msgr2=0x7f2da40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2da40a1380 0x7f2da40a3830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2db810cc70 tx=0x7f2da800b540 comp rx=0 tx=0).stop 2023-07-14T19:39:41.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 msgr2=0x7f2db81aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 0x7f2db81aaa70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2db000b840 tx=0x7f2db000bb50 comp rx=0 tx=0).stop 2023-07-14T19:39:41.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 shutdown_connections 2023-07-14T19:39:41.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f2da40a1380 0x7f2da40a3830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.315 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2db810e330 0x7f2db81a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2db806bbc0 0x7f2db81aaa70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 --2- 172.21.15.82:0/1648595688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db806c630 0x7f2db81aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 >> 172.21.15.82:0/1648595688 conn(0x7f2db806b1c0 msgr2=0x7f2db806f350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:41.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.292+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 shutdown_connections 2023-07-14T19:39:41.316 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.293+0000 7f2dc0711700 1 -- 172.21.15.82:0/1648595688 wait complete. 2023-07-14T19:39:41.317 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 48 2023-07-14T19:39:41.333 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-14T19:39:41.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:41 smithi133 ceph-mon[128894]: pgmap v348: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-14T19:39:41.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:41 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/644046443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:39:41.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:41 smithi082 ceph-mon[127461]: pgmap v348: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-14T19:39:41.487 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:41 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/644046443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:39:41.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.488+0000 7f778168c700 1 -- 172.21.15.82:0/3595154648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 msgr2=0x7f777c0fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.488+0000 7f778168c700 1 --2- 172.21.15.82:0/3595154648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c0fecf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f777400b0d0 tx=0x7f777400b3e0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.489+0000 7f778168c700 1 -- 172.21.15.82:0/3595154648 shutdown_connections 2023-07-14T19:39:41.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.489+0000 7f778168c700 1 --2- 172.21.15.82:0/3595154648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c0ff230 0x7f777c10b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.489+0000 7f778168c700 1 --2- 172.21.15.82:0/3595154648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c103280 0x7f777c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.489+0000 7f778168c700 1 --2- 172.21.15.82:0/3595154648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c0fecf0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.489+0000 7f778168c700 1 -- 172.21.15.82:0/3595154648 >> 172.21.15.82:0/3595154648 conn(0x7f777c074db0 msgr2=0x7f777c0751b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:41.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.490+0000 7f778168c700 1 -- 172.21.15.82:0/3595154648 shutdown_connections 2023-07-14T19:39:41.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.490+0000 7f778168c700 1 -- 172.21.15.82:0/3595154648 wait complete. 2023-07-14T19:39:41.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:41 smithi161 ceph-mon[129184]: pgmap v348: 97 pgs: 31 active+clean+snaptrim_wait, 13 active+clean+snaptrim, 53 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-14T19:39:41.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:41 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/644046443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:39:41.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.490+0000 7f778168c700 1 Processor -- start 2023-07-14T19:39:41.514 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 -- start start 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0ff230 0x7f777c10ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c103280 0x7f777c10f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c10f980 con 0x7f777c0fe880 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c10faf0 con 0x7f777c0ff230 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f778168c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c198500 con 0x7f777c103280 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f777affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:41.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f777affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45182/0 (socket says 172.21.15.82:45182) 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.491+0000 7f777affd700 1 -- 172.21.15.82:0/3932800041 learned_addr learned my addr 172.21.15.82:0/3932800041 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777a7fc700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0ff230 0x7f777c10ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777b7fe700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c103280 0x7f777c10f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777affd700 1 -- 172.21.15.82:0/3932800041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c103280 msgr2=0x7f777c10f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777affd700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c103280 0x7f777c10f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777affd700 1 -- 172.21.15.82:0/3932800041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0ff230 msgr2=0x7f777c10ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.516 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777affd700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0ff230 0x7f777c10ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777affd700 1 -- 172.21.15.82:0/3932800041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770009520 con 0x7f777c0fe880 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f777affd700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c19aa60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f777000eb30 tx=0x7f777000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770009880 con 0x7f777c0fe880 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7774009d00 con 0x7f777c0fe880 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77700044e0 con 0x7f777c0fe880 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770003d80 con 0x7f777c0fe880 2023-07-14T19:39:41.517 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.492+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c198be0 con 0x7f777c0fe880 2023-07-14T19:39:41.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.494+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7768005320 con 0x7f777c0fe880 2023-07-14T19:39:41.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.496+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f777000cbd0 con 0x7f777c0fe880 2023-07-14T19:39:41.518 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.496+0000 7f7763fff700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f77640a1340 0x7f77640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:41.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.497+0000 7f777a7fc700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f77640a1340 0x7f77640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:41.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.497+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f7770014070 con 0x7f777c0fe880 2023-07-14T19:39:41.519 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.497+0000 7f777a7fc700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f77640a1340 0x7f77640a37f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f777c195140 tx=0x7f777400b040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:41.523 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.501+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f777009a070 con 0x7f777c0fe880 2023-07-14T19:39:41.841 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.820+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7768005190 con 0x7f777c0fe880 2023-07-14T19:39:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.823+0000 7f7763fff700 1 -- 172.21.15.82:0/3932800041 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+22900 (secure 0 0 0) 0x7f777007f990 con 0x7f777c0fe880 2023-07-14T19:39:41.847 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-07-14T19:39:41.848 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":96,"fsid":"4f3eb35e-227b-11ee-9b33-001a4aab830c","created":"2023-07-14T19:20:12.915999+0000","modified":"2023-07-14T19:39:11.569714+0000","last_up_change":"2023-07-14T19:26:16.065586+0000","last_in_change":"2023-07-14T19:25:58.245900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-14T19:23:08.979526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-14T19:26:54.722744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-14T19:26:56.092558+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac560fc6-6f94-4da3-bcfa-f4f13ea7b6ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6803","nonce":2396885961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6805","nonce":2396885961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6809","nonce":2396885961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2396885961},{"type":"v1","addr":"172.21.15.82:6807","nonce":2396885961}]},"public_addr":"172.21.15.82:6803/2396885961","cluster_addr":"172.21.15.82:6805/2396885961","heartbeat_back_addr":"172.21.15.82:6809/2396885961","heartbeat_front_addr":"172.21.15.82:6807/2396885961","state":["exists","up"]},{"osd":1,"uuid":"087df2cd-4b6e-4ca5-a4ac-13083d804ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6801","nonce":1413395016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6803","nonce":1413395016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6807","nonce":1413395016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1413395016},{"type":"v1","addr":"172.21.15.133:6805","nonce":1413395016}]},"public_addr":"172.21.15.133:6801/1413395016","cluster_addr":"172.21.15.133:6803/1413395016","heartbeat_back_addr":"172.21.15.133:6807/1413395016","heartbeat_front_addr":"172.21.15.133:6805/1413395016","state":["exists","up"]},{"osd":2,"uuid":"359e8386-59e5-4ffe-9e8a-4e2e6f1f94a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6801","nonce":4228439619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6803","nonce":4228439619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6807","nonce":4228439619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":4228439619},{"type":"v1","addr":"172.21.15.161:6805","nonce":4228439619}]},"public_addr":"172.21.15.161:6801/4228439619","cluster_addr":"172.21.15.161:6803/4228439619","heartbeat_back_addr":"172.21.15.161:6807/4228439619","heartbeat_front_addr":"172.21.15.161:6805/4228439619","state":["exists","up"]},{"osd":3,"uuid":"aabce84f-f9f4-45cc-b738-7414623bae2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6811","nonce":595190701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6813","nonce":595190701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6817","nonce":595190701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":595190701},{"type":"v1","addr":"172.21.15.82:6815","nonce":595190701}]},"public_addr":"172.21.15.82:6811/595190701","cluster_addr":"172.21.15.82:6813/595190701","heartbeat_back_addr":"172.21.15.82:6817/595190701","heartbeat_front_addr":"172.21.15.82:6815/595190701","state":["exists","up"]},{"osd":4,"uuid":"f44de9db-95a0-4007-8acb-40ac3ed8161e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6809","nonce":3870116585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6811","nonce":3870116585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6815","nonce":3870116585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3870116585},{"type":"v1","addr":"172.21.15.133:6813","nonce":3870116585}]},"public_addr":"172.21.15.133:6809/3870116585","cluster_addr":"172.21.15.133:6811/3870116585","heartbeat_back_addr":"172.21.15.133:6815/3870116585","heartbeat_front_addr":"172.21.15.133:6813/3870116585","state":["exists","up"]},{"osd":5,"uuid":"351f3277-08c4-46e3-b1ae-1f78290f556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6809","nonce":4025083410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6811","nonce":4025083410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6815","nonce":4025083410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":4025083410},{"type":"v1","addr":"172.21.15.161:6813","nonce":4025083410}]},"public_addr":"172.21.15.161:6809/4025083410","cluster_addr":"172.21.15.161:6811/4025083410","heartbeat_back_addr":"172.21.15.161:6815/4025083410","heartbeat_front_addr":"172.21.15.161:6813/4025083410","state":["exists","up"]},{"osd":6,"uuid":"69302d50-5480-44c9-815d-c580832c5198","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6819","nonce":2376027432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6821","nonce":2376027432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6825","nonce":2376027432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":2376027432},{"type":"v1","addr":"172.21.15.82:6823","nonce":2376027432}]},"public_addr":"172.21.15.82:6819/2376027432","cluster_addr":"172.21.15.82:6821/2376027432","heartbeat_back_addr":"172.21.15.82:6825/2376027432","heartbeat_front_addr":"172.21.15.82:6823/2376027432","state":["exists","up"]},{"osd":7,"uuid":"a4fa7360-353c-455e-a2d5-4fe907ce87ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6817","nonce":3133253156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6819","nonce":3133253156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6823","nonce":3133253156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3133253156},{"type":"v1","addr":"172.21.15.133:6821","nonce":3133253156}]},"public_addr":"172.21.15.133:6817/3133253156","cluster_addr":"172.21.15.133:6819/3133253156","heartbeat_back_addr":"172.21.15.133:6823/3133253156","heartbeat_front_addr":"172.21.15.133:6821/3133253156","state":["exists","up"]},{"osd":8,"uuid":"16b227e3-291b-4c62-943f-9262f1114940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6817","nonce":119023360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6819","nonce":119023360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6823","nonce":119023360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":119023360},{"type":"v1","addr":"172.21.15.161:6821","nonce":119023360}]},"public_addr":"172.21.15.161:6817/119023360","cluster_addr":"172.21.15.161:6819/119023360","heartbeat_back_addr":"172.21.15.161:6823/119023360","heartbeat_front_addr":"172.21.15.161:6821/119023360","state":["exists","up"]},{"osd":9,"uuid":"300cc59c-8947-4ec0-9352-784783f12ff2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6827","nonce":3744771950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6829","nonce":3744771950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6833","nonce":3744771950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3744771950},{"type":"v1","addr":"172.21.15.82:6831","nonce":3744771950}]},"public_addr":"172.21.15.82:6827/3744771950","cluster_addr":"172.21.15.82:6829/3744771950","heartbeat_back_addr":"172.21.15.82:6833/3744771950","heartbeat_front_addr":"172.21.15.82:6831/3744771950","state":["exists","up"]},{"osd":10,"uuid":"13ec79ce-2a2b-4930-83e8-98530ae6fe96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6825","nonce":2098424958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6827","nonce":2098424958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6831","nonce":2098424958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2098424958},{"type":"v1","addr":"172.21.15.133:6829","nonce":2098424958}]},"public_addr":"172.21.15.133:6825/2098424958","cluster_addr":"172.21.15.133:6827/2098424958","heartbeat_back_addr":"172.21.15.133:6831/2098424958","heartbeat_front_addr":"172.21.15.133:6829/2098424958","state":["exists","up"]},{"osd":11,"uuid":"9803dc37-4330-4d09-b2db-38f8fac795c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6825","nonce":2831052737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6827","nonce":2831052737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6831","nonce":2831052737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2831052737},{"type":"v1","addr":"172.21.15.161:6829","nonce":2831052737}]},"public_addr":"172.21.15.161:6825/2831052737","cluster_addr":"172.21.15.161:6827/2831052737","heartbeat_back_addr":"172.21.15.161:6831/2831052737","heartbeat_front_addr":"172.21.15.161:6829/2831052737","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:24.635823+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:22:45.061221+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:06.172102+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:27.505892+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:23:47.930301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:08.848450+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:29.769953+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:24:50.228227+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:11.552073+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:32.457047+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:25:53.272452+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-14T19:26:14.242961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6801/504215117":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/3311699035":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/1722617918":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/3623265050":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/2410218140":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/1617843074":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/1257676906":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6800/1035467551":"2023-07-15T19:21:04.644444+0000","172.21.15.82:6801/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/504215117":"2023-07-15T19:28:33.481511+0000","172.21.15.82:0/998146971":"2023-07-15T19:20:42.182360+0000","172.21.15.82:6800/3383156270":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/690674119":"2023-07-15T19:20:42.182360+0000","172.21.15.82:0/3474653214":"2023-07-15T19:21:04.644444+0000","172.21.15.82:0/10878106":"2023-07-15T19:20:42.182360+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-14T19:39:41.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f77640a1340 msgr2=0x7f77640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f77640a1340 0x7f77640a37f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f777c195140 tx=0x7f777400b040 comp rx=0 tx=0).stop 2023-07-14T19:39:41.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 msgr2=0x7f777c19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:41.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c19aa60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f777000eb30 tx=0x7f777000ee40 comp rx=0 tx=0).stop 2023-07-14T19:39:41.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 shutdown_connections 2023-07-14T19:39:41.851 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f77640a1340 0x7f77640a37f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777c103280 0x7f777c10f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f777c0fe880 0x7f777c19aa60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 --2- 172.21.15.82:0/3932800041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0ff230 0x7f777c10ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:41.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.829+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 >> 172.21.15.82:0/3932800041 conn(0x7f777c074db0 msgr2=0x7f777c0fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:41.852 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.831+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 shutdown_connections 2023-07-14T19:39:41.856 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:41.835+0000 7f778168c700 1 -- 172.21.15.82:0/3932800041 wait complete. 2023-07-14T19:39:41.877 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-14T19:39:42.050 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 -- 172.21.15.82:0/213735734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 msgr2=0x7f445c10b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 --2- 172.21.15.82:0/213735734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c10b100 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f445400b0a0 tx=0x7f445400b3b0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 -- 172.21.15.82:0/213735734 shutdown_connections 2023-07-14T19:39:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 --2- 172.21.15.82:0/213735734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c10b100 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 --2- 172.21.15.82:0/213735734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1042d0 0x7f445c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.051 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 --2- 172.21.15.82:0/213735734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.028+0000 7f446372d700 1 -- 172.21.15.82:0/213735734 >> 172.21.15.82:0/213735734 conn(0x7f445c075190 msgr2=0x7f445c075590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.029+0000 7f446372d700 1 -- 172.21.15.82:0/213735734 shutdown_connections 2023-07-14T19:39:42.052 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.029+0000 7f446372d700 1 -- 172.21.15.82:0/213735734 wait complete. 2023-07-14T19:39:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.029+0000 7f446372d700 1 Processor -- start 2023-07-14T19:39:42.053 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 -- start start 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1042d0 0x7f445c107cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c108490 con 0x7f445c0fed60 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c108600 con 0x7f445c0fe3d0 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f446372d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c1087c0 con 0x7f445c1042d0 2023-07-14T19:39:42.054 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f44614c9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4460cc8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f44614c9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41542/0 (socket says 172.21.15.82:41542) 2023-07-14T19:39:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f44614c9700 1 -- 172.21.15.82:0/1331504267 learned_addr learned my addr 172.21.15.82:0/1331504267 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4461cca700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1042d0 0x7f445c107cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4460cc8700 1 -- 172.21.15.82:0/1331504267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1042d0 msgr2=0x7f445c107cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.055 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4460cc8700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1042d0 0x7f445c107cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4460cc8700 1 -- 172.21.15.82:0/1331504267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 msgr2=0x7f445c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4460cc8700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f4460cc8700 1 -- 172.21.15.82:0/1331504267 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454009d00 con 0x7f445c0fed60 2023-07-14T19:39:42.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.030+0000 7f44614c9700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:39:42.056 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.031+0000 7f4460cc8700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c198f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f445800e9f0 tx=0x7f445800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:42.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.032+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800c9c0 con 0x7f445c0fed60 2023-07-14T19:39:42.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.032+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4458010430 con 0x7f445c0fed60 2023-07-14T19:39:42.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.032+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458018680 con 0x7f445c0fed60 2023-07-14T19:39:42.057 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.034+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c108aa0 con 0x7f445c0fed60 2023-07-14T19:39:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.034+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c19ca00 con 0x7f445c0fed60 2023-07-14T19:39:42.058 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.035+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c067810 con 0x7f445c0fed60 2023-07-14T19:39:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.037+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4458010860 con 0x7f445c0fed60 2023-07-14T19:39:42.059 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.038+0000 7f444e7fc700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f44480a1340 0x7f44480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.038+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8470+0+0 (secure 0 0 0) 0x7f4458014070 con 0x7f445c0fed60 2023-07-14T19:39:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.038+0000 7f44614c9700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f44480a1340 0x7f44480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.060 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.038+0000 7f44614c9700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f44480a1340 0x7f44480a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4450005950 tx=0x7f445000a400 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:42.063 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.042+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f4458096bc0 con 0x7f445c0fed60 2023-07-14T19:39:42.439 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.417+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f445c19ce70 con 0x7f445c0fed60 2023-07-14T19:39:42.451 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.429+0000 7f444e7fc700 1 -- 172.21.15.82:0/1331504267 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7f445809a1e0 con 0x7f445c0fed60 2023-07-14T19:39:42.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-mon[128894]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:39:42.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1648595688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:39:42.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/3932800041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:39:42.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f44480a1340 msgr2=0x7f44480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f44480a1340 0x7f44480a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4450005950 tx=0x7f445000a400 comp rx=0 tx=0).stop 2023-07-14T19:39:42.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 msgr2=0x7f445c198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.455 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c198f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f445800e9f0 tx=0x7f445800ed00 comp rx=0 tx=0).stop 2023-07-14T19:39:42.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 shutdown_connections 2023-07-14T19:39:42.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f44480a1340 0x7f44480a37f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f445c1042d0 0x7f445c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0fe3d0 0x7f445c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.456 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 --2- 172.21.15.82:0/1331504267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0fed60 0x7f445c198f50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.433+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 >> 172.21.15.82:0/1331504267 conn(0x7f445c075190 msgr2=0x7f445c10c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:42.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.435+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 shutdown_connections 2023-07-14T19:39:42.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.435+0000 7f446372d700 1 -- 172.21.15.82:0/1331504267 wait complete. 2023-07-14T19:39:42.457 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-14T19:39:42.468 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:42 smithi082 ceph-mon[127461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:39:42.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:42 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1648595688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:39:42.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:42 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/3932800041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:39:42.469 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:39:42.418+0000 7f9a2b47e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T19:39:42.470 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-14T19:39:42.511 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-mon[129184]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-14T19:39:42.511 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1648595688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-14T19:39:42.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/3932800041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-14T19:39:42.512 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f[154624]: ignoring --setuser ceph since I am not root 2023-07-14T19:39:42.512 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f[154624]: ignoring --setgroup ceph since I am not root 2023-07-14T19:39:42.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.634+0000 7f061dbd5700 1 -- 172.21.15.82:0/3598809527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06180ff4b0 msgr2=0x7f061810b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.634+0000 7f061dbd5700 1 --2- 172.21.15.82:0/3598809527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06180ff4b0 0x7f061810b940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600009ca0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 -- 172.21.15.82:0/3598809527 shutdown_connections 2023-07-14T19:39:42.657 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 --2- 172.21.15.82:0/3598809527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06180ff4b0 0x7f061810b940 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 --2- 172.21.15.82:0/3598809527 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0618104b10 0x7f0618104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 --2- 172.21.15.82:0/3598809527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f06180fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 -- 172.21.15.82:0/3598809527 >> 172.21.15.82:0/3598809527 conn(0x7f06180fa5e0 msgr2=0x7f06180fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:42.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 -- 172.21.15.82:0/3598809527 shutdown_connections 2023-07-14T19:39:42.658 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 -- 172.21.15.82:0/3598809527 wait complete. 2023-07-14T19:39:42.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.635+0000 7f061dbd5700 1 Processor -- start 2023-07-14T19:39:42.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 -- start start 2023-07-14T19:39:42.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f061819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.660 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06180ff4b0 0x7f061819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0618104b10 0x7f0618194a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0618195070 con 0x7f0618104b10 2023-07-14T19:39:42.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06181951e0 con 0x7f06180feb00 2023-07-14T19:39:42.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f061dbd5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06181953a0 con 0x7f06180ff4b0 2023-07-14T19:39:42.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f06177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f061819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f0616ffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06180ff4b0 0x7f061819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f06177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f061819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.82:41556/0 (socket says 172.21.15.82:41556) 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f06177fe700 1 -- 172.21.15.82:0/1409134721 learned_addr learned my addr 172.21.15.82:0/1409134721 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.636+0000 7f06177fe700 1 -- 172.21.15.82:0/1409134721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06180ff4b0 msgr2=0x7f061819aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f06177fe700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06180ff4b0 0x7f061819aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f06177fe700 1 -- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0618104b10 msgr2=0x7f0618194a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f06177fe700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0618104b10 0x7f0618194a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f06177fe700 1 -- 172.21.15.82:0/1409134721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000095f0 con 0x7f06180feb00 2023-07-14T19:39:42.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f0616ffd700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06180ff4b0 0x7f061819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-14T19:39:42.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f06177fe700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f061819a990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f061000ef10 tx=0x7f061000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:42.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000ca70 con 0x7f06180feb00 2023-07-14T19:39:42.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061000f040 con 0x7f06180feb00 2023-07-14T19:39:42.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0618195620 con 0x7f06180feb00 2023-07-14T19:39:42.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100146e0 con 0x7f06180feb00 2023-07-14T19:39:42.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.637+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0618067980 con 0x7f06180feb00 2023-07-14T19:39:42.664 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.638+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06180673c0 con 0x7f06180feb00 2023-07-14T19:39:42.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.640+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0610007500 con 0x7f06180feb00 2023-07-14T19:39:42.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.641+0000 7f0614ff9700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f06040a12c0 0x7f06040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:42.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.641+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8684+0+0 (secure 0 0 0) 0x7f06100cd020 con 0x7f06180feb00 2023-07-14T19:39:42.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.641+0000 7f0616ffd700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f06040a12c0 0x7f06040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:42.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.641+0000 7f0616ffd700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f06040a12c0 0x7f06040a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0618104080 tx=0x7f0608012040 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:42.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:42.647+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f06100963b0 con 0x7f06180feb00 2023-07-14T19:39:42.762 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: ignoring --setuser ceph since I am not root 2023-07-14T19:39:42.762 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: ignoring --setgroup ceph since I am not root 2023-07-14T19:39:42.762 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: starting mds.c at 2023-07-14T19:39:42.763 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f[154624]: starting mds.f at 2023-07-14T19:39:42.763 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-i[155299]: ignoring --setuser ceph since I am not root 2023-07-14T19:39:42.763 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-i[155299]: ignoring --setgroup ceph since I am not root 2023-07-14T19:39:42.764 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-mds[155323]: -- 172.21.15.161:0/3565293859 <== mon.0 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56151d7871e0 con 0x56151d75cc00 2023-07-14T19:39:42.764 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:39:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-i[155299]: starting mds.i at 2023-07-14T19:39:42.954 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: ignoring --setuser ceph since I am not root 2023-07-14T19:39:42.955 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: ignoring --setgroup ceph since I am not root 2023-07-14T19:39:42.955 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: starting mds.b at 2023-07-14T19:39:42.955 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e[153424]: ignoring --setuser ceph since I am not root 2023-07-14T19:39:42.956 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e[153424]: ignoring --setgroup ceph since I am not root 2023-07-14T19:39:42.956 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-mds[153448]: -- 172.21.15.133:0/2817124534 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558fca1011e0 con 0x558fca0d6c00 2023-07-14T19:39:42.956 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e[153424]: starting mds.e at 2023-07-14T19:39:42.957 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: ignoring --setuser ceph since I am not root 2023-07-14T19:39:42.957 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: ignoring --setgroup ceph since I am not root 2023-07-14T19:39:42.958 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-mds[153790]: -- 172.21.15.133:0/1486423129 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561fc1def1e0 con 0x561fc1dc4800 2023-07-14T19:39:42.958 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:39:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: starting mds.h at 2023-07-14T19:39:43.029 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.007+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0618067ce0 con 0x7f06180feb00 2023-07-14T19:39:43.452 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.430+0000 7f0614ff9700 1 -- 172.21.15.82:0/1409134721 <== mon.2 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f06100999d0 con 0x7f06180feb00 2023-07-14T19:39:43.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: pgmap v349: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2023-07-14T19:39:43.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1331504267' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T19:39:43.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T19:39:43.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T19:39:43.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-14T19:39:43.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: osdmap e97: 12 total, 12 up, 12 in 2023-07-14T19:39:43.455 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1331504267' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T19:39:43.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T19:39:43.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/1409134721' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:39:43.456 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:43 smithi133 ceph-mon[128894]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:39:43.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.433+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f06040a12c0 msgr2=0x7f06040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:43.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.433+0000 7f061dbd5700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f06040a12c0 0x7f06040a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0618104080 tx=0x7f0608012040 comp rx=0 tx=0).stop 2023-07-14T19:39:43.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.433+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 msgr2=0x7f061819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:43.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.433+0000 7f061dbd5700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f061819a990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f061000ef10 tx=0x7f061000c3a0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 shutdown_connections 2023-07-14T19:39:43.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f06040a12c0 0x7f06040a3770 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0618104080 tx=0x7f0608012040 comp rx=0 tx=0).stop 2023-07-14T19:39:43.458 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0618104b10 0x7f0618194a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06180feb00 0x7f061819a990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 --2- 172.21.15.82:0/1409134721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06180ff4b0 0x7f061819aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.459 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 >> 172.21.15.82:0/1409134721 conn(0x7f06180fa5e0 msgr2=0x7f06180fab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:43.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.434+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 shutdown_connections 2023-07-14T19:39:43.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.435+0000 7f061dbd5700 1 -- 172.21.15.82:0/1409134721 wait complete. 2023-07-14T19:39:43.477 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-14T19:39:43.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: pgmap v349: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2023-07-14T19:39:43.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1331504267' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T19:39:43.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T19:39:43.478 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T19:39:43.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-14T19:39:43.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: osdmap e97: 12 total, 12 up, 12 in 2023-07-14T19:39:43.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1331504267' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T19:39:43.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T19:39:43.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/1409134721' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:39:43.479 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:43 smithi082 ceph-mon[127461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:39:43.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: pgmap v349: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2023-07-14T19:39:43.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1331504267' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-14T19:39:43.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: osdmap e97: 12 total, 12 up, 12 in 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1331504267' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/1409134721' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:39:43.513 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:43 smithi161 ceph-mon[129184]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-14T19:39:43.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.645+0000 7f1f101d6700 1 -- 172.21.15.82:0/2959489099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0806d4d0 msgr2=0x7f1f0811b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:43.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.645+0000 7f1f101d6700 1 --2- 172.21.15.82:0/2959489099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0806d4d0 0x7f1f0811b5a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f0001c0d0 tx=0x7f1f0001c3e0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.646+0000 7f1f101d6700 1 -- 172.21.15.82:0/2959489099 shutdown_connections 2023-07-14T19:39:43.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.646+0000 7f1f101d6700 1 --2- 172.21.15.82:0/2959489099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0806d4d0 0x7f1f0811b5a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.646+0000 7f1f101d6700 1 --2- 172.21.15.82:0/2959489099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0810e3b0 0x7f1f0810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.646+0000 7f1f101d6700 1 --2- 172.21.15.82:0/2959489099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f0806cad0 0x7f1f0806cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.646+0000 7f1f101d6700 1 -- 172.21.15.82:0/2959489099 >> 172.21.15.82:0/2959489099 conn(0x7f1f0806c0a0 msgr2=0x7f1f08071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:43.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.648+0000 7f1f101d6700 1 -- 172.21.15.82:0/2959489099 shutdown_connections 2023-07-14T19:39:43.671 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.648+0000 7f1f101d6700 1 -- 172.21.15.82:0/2959489099 wait complete. 2023-07-14T19:39:43.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.649+0000 7f1f101d6700 1 Processor -- start 2023-07-14T19:39:43.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.649+0000 7f1f101d6700 1 -- start start 2023-07-14T19:39:43.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.649+0000 7f1f101d6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f0806cad0 0x7f1f08116e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:43.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f101d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 0x7f1f08117380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:43.673 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f101d6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0810e3b0 0x7f1f081a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f101d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a85e0 con 0x7f1f0806d4d0 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f101d6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a8750 con 0x7f1f0806cad0 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f101d6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a8910 con 0x7f1f0810e3b0 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 0x7f1f08117380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 0x7f1f08117380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45220/0 (socket says 172.21.15.82:45220) 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 -- 172.21.15.82:0/4065930303 learned_addr learned my addr 172.21.15.82:0/4065930303 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:43.674 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0df72700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f0806cad0 0x7f1f08116e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0e773700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0810e3b0 0x7f1f081a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 -- 172.21.15.82:0/4065930303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0810e3b0 msgr2=0x7f1f081a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0810e3b0 0x7f1f081a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 -- 172.21.15.82:0/4065930303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f0806cad0 msgr2=0x7f1f08116e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f0806cad0 0x7f1f08116e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 -- 172.21.15.82:0/4065930303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0001ad10 con 0x7f1f0806d4d0 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.650+0000 7f1f0d771700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 0x7f1f08117380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400e9a0 tx=0x7f1f0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:43.675 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.651+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04009790 con 0x7f1f0806d4d0 2023-07-14T19:39:43.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.651+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f081a8bf0 con 0x7f1f0806d4d0 2023-07-14T19:39:43.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.651+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f04010430 con 0x7f1f0806d4d0 2023-07-14T19:39:43.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.651+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04018680 con 0x7f1f0806d4d0 2023-07-14T19:39:43.676 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.651+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081a9150 con 0x7f1f0806d4d0 2023-07-14T19:39:43.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.652+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f080673c0 con 0x7f1f0806d4d0 2023-07-14T19:39:43.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.653+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1f040105a0 con 0x7f1f0806d4d0 2023-07-14T19:39:43.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.654+0000 7f1efeffd700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1ef40a12c0 0x7f1ef40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:43.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.654+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8684+0+0 (secure 0 0 0) 0x7f1f04014070 con 0x7f1f0806d4d0 2023-07-14T19:39:43.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.654+0000 7f1f0df72700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1ef40a12c0 0x7f1ef40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:43.677 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.655+0000 7f1f0df72700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1ef40a12c0 0x7f1ef40a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8006fd0 tx=0x7f1ef8009190 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:43.681 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.659+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f1f04096b70 con 0x7f1f0806d4d0 2023-07-14T19:39:43.971 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:43.949+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f08118190 con 0x7f1f0806d4d0 2023-07-14T19:39:44.457 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.435+0000 7f1efeffd700 1 -- 172.21.15.82:0/4065930303 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1f040b8e60 con 0x7f1f0806d4d0 2023-07-14T19:39:44.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.443+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1ef40a12c0 msgr2=0x7f1ef40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:44.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.443+0000 7f1f101d6700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1ef40a12c0 0x7f1ef40a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8006fd0 tx=0x7f1ef8009190 comp rx=0 tx=0).stop 2023-07-14T19:39:44.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.443+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 msgr2=0x7f1f08117380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:44.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.443+0000 7f1f101d6700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 0x7f1f08117380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1f0400e9a0 tx=0x7f1f0400ecb0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.447+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 shutdown_connections 2023-07-14T19:39:44.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.447+0000 7f1f101d6700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f1ef40a12c0 0x7f1ef40a3770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.447+0000 7f1f101d6700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f0810e3b0 0x7f1f081a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.469 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.447+0000 7f1f101d6700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f0806cad0 0x7f1f08116e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.447+0000 7f1f101d6700 1 --2- 172.21.15.82:0/4065930303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f0806d4d0 0x7f1f08117380 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.470 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.447+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 >> 172.21.15.82:0/4065930303 conn(0x7f1f0806c0a0 msgr2=0x7f1f0806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.451+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 shutdown_connections 2023-07-14T19:39:44.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.451+0000 7f1f101d6700 1 -- 172.21.15.82:0/4065930303 wait complete. 2023-07-14T19:39:44.487 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2023-07-14T19:39:44.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-14T19:39:44.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 -- 172.21.15.82:0/30361058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fe5a0 msgr2=0x7f0b700fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:44.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/30361058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fe5a0 0x7f0b700fe970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400aad0 tx=0x7f0b6400ade0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 -- 172.21.15.82:0/30361058 shutdown_connections 2023-07-14T19:39:44.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/30361058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b70103560 0x7f0b7010b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.686 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/30361058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fe5a0 0x7f0b700fe970 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/30361058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b700fef40 0x7f0b70102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 -- 172.21.15.82:0/30361058 >> 172.21.15.82:0/30361058 conn(0x7f0b70076330 msgr2=0x7f0b70076730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:44.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 -- 172.21.15.82:0/30361058 shutdown_connections 2023-07-14T19:39:44.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.663+0000 7f0b76ab6700 1 -- 172.21.15.82:0/30361058 wait complete. 2023-07-14T19:39:44.687 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 Processor -- start 2023-07-14T19:39:44.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 -- start start 2023-07-14T19:39:44.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700fe5a0 0x7f0b70195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:44.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 0x7f0b70196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:44.689 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b70103560 0x7f0b7019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7019adf0 con 0x7f0b700fef40 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7019af60 con 0x7f0b70103560 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b76ab6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7019b120 con 0x7f0b700fe5a0 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b6ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 0x7f0b70196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b6ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 0x7f0b70196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45234/0 (socket says 172.21.15.82:45234) 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b6ffff700 1 -- 172.21.15.82:0/406739646 learned_addr learned my addr 172.21.15.82:0/406739646 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-07-14T19:39:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.664+0000 7f0b75053700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b70103560 0x7f0b7019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b74852700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700fe5a0 0x7f0b70195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6ffff700 1 -- 172.21.15.82:0/406739646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700fe5a0 msgr2=0x7f0b70195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6ffff700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700fe5a0 0x7f0b70195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6ffff700 1 -- 172.21.15.82:0/406739646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b70103560 msgr2=0x7f0b7019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6ffff700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b70103560 0x7f0b7019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6ffff700 1 -- 172.21.15.82:0/406739646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6400a700 con 0x7f0b700fef40 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6ffff700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 0x7f0b70196350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000e940 tx=0x7f0b6000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:44.691 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60009730 con 0x7f0b700fef40 2023-07-14T19:39:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b60010430 con 0x7f0b700fef40 2023-07-14T19:39:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7019b400 con 0x7f0b700fef40 2023-07-14T19:39:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6000cea0 con 0x7f0b700fef40 2023-07-14T19:39:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.665+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b701a2c40 con 0x7f0b700fef40 2023-07-14T19:39:44.692 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.669+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0b60010810 con 0x7f0b700fef40 2023-07-14T19:39:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.669+0000 7f0b6dffb700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0b580a1340 0x7f0b580a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-14T19:39:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.669+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8188+0+0 (secure 0 0 0) 0x7f0b60014070 con 0x7f0b700fef40 2023-07-14T19:39:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.670+0000 7f0b74852700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0b580a1340 0x7f0b580a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-14T19:39:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.670+0000 7f0b74852700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0b580a1340 0x7f0b580a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400afd0 tx=0x7f0b64005fb0 comp rx=0 tx=0).ready entity=mgr.15414 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-14T19:39:44.695 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.674+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b70108800 con 0x7f0b700fef40 2023-07-14T19:39:44.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.682+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178025 (secure 0 0 0) 0x7f0b60051780 con 0x7f0b700fef40 2023-07-14T19:39:44.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T19:39:44.704 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: Cluster is now healthy 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6838/2887495567,v1:172.21.15.161:6839/2887495567] up:boot 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6834/3684198326,v1:172.21.15.161:6835/3684198326] up:boot 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.161:6836/2514050148,v1:172.21.15.161:6837/2514050148] up:boot 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6836/3865468208,v1:172.21.15.133:6837/3865468208] up:boot 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6838/294008447,v1:172.21.15.133:6839/294008447] up:boot 2023-07-14T19:39:44.705 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: mds.? [v2:172.21.15.133:6834/4004937387,v1:172.21.15.133:6835/4004937387] up:boot 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: fsmap 12 up:standby 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:39:44.706 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:39:44.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:39:44.707 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:44 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4065930303' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T19:39:44.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T19:39:44.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T19:39:44.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: Cluster is now healthy 2023-07-14T19:39:44.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6838/2887495567,v1:172.21.15.161:6839/2887495567] up:boot 2023-07-14T19:39:44.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6834/3684198326,v1:172.21.15.161:6835/3684198326] up:boot 2023-07-14T19:39:44.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.161:6836/2514050148,v1:172.21.15.161:6837/2514050148] up:boot 2023-07-14T19:39:44.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6836/3865468208,v1:172.21.15.133:6837/3865468208] up:boot 2023-07-14T19:39:44.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6838/294008447,v1:172.21.15.133:6839/294008447] up:boot 2023-07-14T19:39:44.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: mds.? [v2:172.21.15.133:6834/4004937387,v1:172.21.15.133:6835/4004937387] up:boot 2023-07-14T19:39:44.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T19:39:44.736 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: fsmap 12 up:standby 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:39:44.737 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:44 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4065930303' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T19:39:44.762 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-14T19:39:44.762 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: Cluster is now healthy 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6838/2887495567,v1:172.21.15.161:6839/2887495567] up:boot 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6834/3684198326,v1:172.21.15.161:6835/3684198326] up:boot 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.161:6836/2514050148,v1:172.21.15.161:6837/2514050148] up:boot 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6836/3865468208,v1:172.21.15.133:6837/3865468208] up:boot 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6838/294008447,v1:172.21.15.133:6839/294008447] up:boot 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: mds.? [v2:172.21.15.133:6834/4004937387,v1:172.21.15.133:6835/4004937387] up:boot 2023-07-14T19:39:44.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: fsmap 12 up:standby 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-14T19:39:44.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='mgr.15414 172.21.15.82:0/1870593233' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-14T19:39:44.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:44 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4065930303' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T19:39:45.003 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:44.981+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b70197160 con 0x7f0b700fef40 2023-07-14T19:39:45.467 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.445+0000 7f0b6dffb700 1 -- 172.21.15.82:0/406739646 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0b60096230 con 0x7f0b700fef40 2023-07-14T19:39:45.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0b580a1340 msgr2=0x7f0b580a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:45.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0b580a1340 0x7f0b580a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400afd0 tx=0x7f0b64005fb0 comp rx=0 tx=0).stop 2023-07-14T19:39:45.473 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 msgr2=0x7f0b70196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 0x7f0b70196350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000e940 tx=0x7f0b6000ec50 comp rx=0 tx=0).stop 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 shutdown_connections 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:6800/2084410629,v1:172.21.15.82:6801/2084410629] conn(0x7f0b580a1340 0x7f0b580a37f0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400afd0 tx=0x7f0b64005fb0 comp rx=0 tx=0).stop 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b70103560 0x7f0b7019a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b700fe5a0 0x7f0b70195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 --2- 172.21.15.82:0/406739646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b700fef40 0x7f0b70196350 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-14T19:39:45.474 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.451+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 >> 172.21.15.82:0/406739646 conn(0x7f0b70076330 msgr2=0x7f0b700fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-14T19:39:45.481 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.456+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 shutdown_connections 2023-07-14T19:39:45.482 INFO:teuthology.orchestra.run.smithi082.stderr:2023-07-14T19:39:45.460+0000 7f0b76ab6700 1 -- 172.21.15.82:0/406739646 wait complete. 2023-07-14T19:39:45.494 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2023-07-14T19:39:45.511 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:39:45.560 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:39:45.627 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:39:45.658 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-14T19:39:45.659 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T19:39:45.683 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T19:39:45.700 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-14T19:39:45.717 INFO:tasks.cephadm:Stopping all daemons... 2023-07-14T19:39:45.717 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-14T19:39:45.718 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a 2023-07-14T19:39:45.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:45 smithi082 ceph-mon[127461]: pgmap v351: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2023-07-14T19:39:45.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:45 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/4065930303' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T19:39:45.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:45 smithi082 ceph-mon[127461]: osdmap e98: 12 total, 12 up, 12 in 2023-07-14T19:39:45.735 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:45 smithi082 ceph-mon[127461]: from='client.? 172.21.15.82:0/406739646' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T19:39:45.762 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:45 smithi161 ceph-mon[129184]: pgmap v351: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2023-07-14T19:39:45.762 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:45 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/4065930303' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T19:39:45.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:45 smithi161 ceph-mon[129184]: osdmap e98: 12 total, 12 up, 12 in 2023-07-14T19:39:45.763 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:45 smithi161 ceph-mon[129184]: from='client.? 172.21.15.82:0/406739646' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T19:39:45.954 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:45 smithi133 ceph-mon[128894]: pgmap v351: 97 pgs: 23 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 66 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 818 B/s wr, 7 op/s 2023-07-14T19:39:45.954 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:45 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/4065930303' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-14T19:39:45.955 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:45 smithi133 ceph-mon[128894]: osdmap e98: 12 total, 12 up, 12 in 2023-07-14T19:39:45.955 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:45 smithi133 ceph-mon[128894]: from='client.? 172.21.15.82:0/406739646' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-14T19:39:46.234 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:45 smithi082 systemd[1]: Stopping Ceph mon.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:46.593 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:39:46.279+0000 7f9a30c89700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:39:46.594 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a[127437]: 2023-07-14T19:39:46.279+0000 7f9a30c89700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-14T19:39:48.924 INFO:journalctl@ceph.mon.a.smithi082.stdout:Jul 14 19:39:48 smithi082 bash[187856]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-a 2023-07-14T19:39:48.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.a.service' 2023-07-14T19:39:49.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:39:49.158 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-14T19:39:49.158 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-14T19:39:49.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.b 2023-07-14T19:39:49.454 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:49 smithi133 systemd[1]: Stopping Ceph mon.b for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:50.039 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-b[128870]: 2023-07-14T19:39:49.738+0000 7fca38dba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:39:50.039 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-b[128870]: 2023-07-14T19:39:49.738+0000 7fca38dba700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-14T19:39:50.340 INFO:journalctl@ceph.mon.b.smithi133.stdout:Jul 14 19:39:50 smithi133 bash[160217]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-b 2023-07-14T19:39:50.525 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.b.service' 2023-07-14T19:39:50.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:39:50.589 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-14T19:39:50.589 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-14T19:39:50.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.c 2023-07-14T19:39:51.012 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:50 smithi161 systemd[1]: Stopping Ceph mon.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:51.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-c[129151]: 2023-07-14T19:39:51.039+0000 7fefb2e3e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:39:51.368 INFO:journalctl@ceph.mon.c.smithi161.stdout:Jul 14 19:39:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mon-c[129151]: 2023-07-14T19:39:51.039+0000 7fefb2e3e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-14T19:39:51.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mon.c.service' 2023-07-14T19:39:51.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:39:51.652 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-14T19:39:51.652 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-14T19:39:51.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x 2023-07-14T19:39:51.985 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:39:51 smithi082 systemd[1]: Stopping Ceph mgr.x for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:52.280 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:39:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:39:51.985+0000 7fad30267700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:39:52.281 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:39:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x[127706]: 2023-07-14T19:39:51.985+0000 7fad30267700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-14T19:39:52.537 INFO:journalctl@ceph.mgr.x.smithi082.stdout:Jul 14 19:39:52 smithi082 bash[187997]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-x 2023-07-14T19:39:52.543 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.x.service' 2023-07-14T19:39:52.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:39:52.580 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-14T19:39:52.581 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-14T19:39:52.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.y 2023-07-14T19:39:52.954 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:39:52 smithi133 systemd[1]: Stopping Ceph mgr.y for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:53.366 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:39:53 smithi133 bash[160361]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-y 2023-07-14T19:39:53.366 INFO:journalctl@ceph.mgr.y.smithi133.stdout:Jul 14 19:39:53 smithi133 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-14T19:39:53.372 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.y.service' 2023-07-14T19:39:53.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:39:53.412 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-14T19:39:53.413 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-14T19:39:53.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.z 2023-07-14T19:39:53.762 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:39:53 smithi161 systemd[1]: Stopping Ceph mgr.z for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:54.131 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Jul 14 19:39:54 smithi161 bash[162170]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mgr-z 2023-07-14T19:39:54.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mgr.z.service' 2023-07-14T19:39:54.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:39:54.351 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-14T19:39:54.352 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-14T19:39:54.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.0 2023-07-14T19:39:54.698 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:39:54 smithi082 systemd[1]: Stopping Ceph osd.0 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:39:54.985 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:39:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0[140455]: 2023-07-14T19:39:54.675+0000 7fa2b27ec700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:39:54.985 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:39:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0[140455]: 2023-07-14T19:39:54.675+0000 7fa2b27ec700 -1 osd.0 99 *** Got signal Terminated *** 2023-07-14T19:39:54.986 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:39:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0[140455]: 2023-07-14T19:39:54.675+0000 7fa2b27ec700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:00.736 INFO:journalctl@ceph.osd.0.smithi082.stdout:Jul 14 19:40:00 smithi082 bash[188136]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-0 2023-07-14T19:40:01.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.0.service' 2023-07-14T19:40:01.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:01.791 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-14T19:40:01.792 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-14T19:40:01.792 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.1 2023-07-14T19:40:02.153 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:40:01 smithi133 systemd[1]: Stopping Ceph osd.1 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:02.455 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:40:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1[133886]: 2023-07-14T19:40:02.130+0000 7fe2fa914700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:02.455 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:40:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1[133886]: 2023-07-14T19:40:02.130+0000 7fe2fa914700 -1 osd.1 99 *** Got signal Terminated *** 2023-07-14T19:40:02.455 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:40:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1[133886]: 2023-07-14T19:40:02.130+0000 7fe2fa914700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:07.932 INFO:journalctl@ceph.osd.1.smithi133.stdout:Jul 14 19:40:07 smithi133 bash[160507]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-1 2023-07-14T19:40:09.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.1.service' 2023-07-14T19:40:09.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:09.238 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-14T19:40:09.238 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-14T19:40:09.239 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.2 2023-07-14T19:40:09.609 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:40:09 smithi161 systemd[1]: Stopping Ceph osd.2 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:10.013 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:40:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2[134957]: 2023-07-14T19:40:09.586+0000 7f3ee7b7a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:10.013 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:40:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2[134957]: 2023-07-14T19:40:09.586+0000 7f3ee7b7a700 -1 osd.2 99 *** Got signal Terminated *** 2023-07-14T19:40:10.013 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:40:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2[134957]: 2023-07-14T19:40:09.586+0000 7f3ee7b7a700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:16.013 INFO:journalctl@ceph.osd.2.smithi161.stdout:Jul 14 19:40:15 smithi161 bash[162321]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-2 2023-07-14T19:40:17.141 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.2.service' 2023-07-14T19:40:17.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:17.185 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-14T19:40:17.185 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-14T19:40:17.185 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.3 2023-07-14T19:40:17.486 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:40:17 smithi082 systemd[1]: Stopping Ceph osd.3 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:17.986 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:40:17 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3[145767]: 2023-07-14T19:40:17.520+0000 7facb9caa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:17.986 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:40:17 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3[145767]: 2023-07-14T19:40:17.520+0000 7facb9caa700 -1 osd.3 99 *** Got signal Terminated *** 2023-07-14T19:40:17.986 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:40:17 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3[145767]: 2023-07-14T19:40:17.520+0000 7facb9caa700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:21.263 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:20.901+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:21.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:21 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:21.456+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:22.263 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:21.876+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:22.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:22.450+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:22.736 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:22 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:22.309+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:23.263 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:22 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:22.831+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:23.320 INFO:journalctl@ceph.osd.3.smithi082.stdout:Jul 14 19:40:23 smithi082 bash[188414]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-3 2023-07-14T19:40:23.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:23 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:23.431+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:23.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:23 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:23.305+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:24.263 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:23 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:23.854+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:24.674 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:24 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:24.435+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:24.674 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:24 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:24.261+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:24.681 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.3.service' 2023-07-14T19:40:24.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:24.720 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-14T19:40:24.720 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-14T19:40:24.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.4 2023-07-14T19:40:25.067 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:40:24 smithi133 systemd[1]: Stopping Ceph osd.4 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:25.116 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:24 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:24.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:25.456 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:40:25 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4[139330]: 2023-07-14T19:40:25.043+0000 7f9a2f760700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:25.456 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:40:25 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4[139330]: 2023-07-14T19:40:25.043+0000 7f9a2f760700 -1 osd.4 99 *** Got signal Terminated *** 2023-07-14T19:40:25.456 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:40:25 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4[139330]: 2023-07-14T19:40:25.043+0000 7f9a2f760700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:25.513 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:25 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:25.094+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:25.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:25.411+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:25.736 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:25.277+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:26.145 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:25 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:25.852+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:26.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:25 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:25.715+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:26.513 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:26 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:26.122+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:26.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:26 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:26.374+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:26.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:26 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:26.241+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:27.143 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:26 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:26.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:27.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:26 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:26.687+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:27.513 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:27.119+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:27.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:27.391+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:27.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:27 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:27.258+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:28.125 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:27.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:28.125 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:27 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:27.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:28.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:27 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:27.732+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:28.513 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:28 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:28.101+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:28.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:28.407+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:28.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:28.269+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:28.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:28 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:28.269+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:29.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:28 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:28.714+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:29.263 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:29 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:29.118+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:29.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:28 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:28.885+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:29.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:28 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:28.885+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:29.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:29 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:29.398+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:29.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:29 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:29.398+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:29.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:29 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:29.251+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:29.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:29 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:29.251+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:30.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:29 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:29.727+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:30.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:29 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:29.727+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:30.263 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:30.076+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:30.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:30.076+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:30.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:29 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:29.845+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:30.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:29 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:29.846+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:30.681 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:30.336+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:30.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:30.367+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:30.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:30.367+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:30.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:30.245+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:30.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:30.245+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:30.956 INFO:journalctl@ceph.osd.4.smithi133.stdout:Jul 14 19:40:30 smithi133 bash[160786]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-4 2023-07-14T19:40:30.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:30.688+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:30.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:30 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:30.688+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:31.263 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:31.049+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:31.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:31.049+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:31.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:30.884+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:31.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:30 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:30.884+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:31.675 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:31 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:31.321+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:31.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:31.374+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:31.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:31.374+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:31.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:31.271+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:31.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:31 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:31.271+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:31.910 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:31.498+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:31.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:31 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:31.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:31.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:31 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:31.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:32.016 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.4.service' 2023-07-14T19:40:32.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:32.055 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-14T19:40:32.056 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-14T19:40:32.056 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.5 2023-07-14T19:40:32.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:31.886+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:32.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:31 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:31.886+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:32.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:32 smithi161 systemd[1]: Stopping Ceph osd.5 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:32.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:32.084+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:32.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:32.084+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:32.664 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:32.331+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:32.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:32.342+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:32.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:32.342+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:32.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:32.305+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:32.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:32 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:32.305+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:32.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:32.505+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:32.764 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:32.407+0000 7f88d54ab700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:32.765 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:32.407+0000 7f88d54ab700 -1 osd.5 99 *** Got signal Terminated *** 2023-07-14T19:40:32.765 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:32.407+0000 7f88d54ab700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:32.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:32.640+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:32.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:32 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:32.640+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:33.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:33 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:33.047+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:33.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:33 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:33.047+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:33.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:32.870+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:33.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:32 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:32.870+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:33.681 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:33 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:33.315+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:33.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:33.353+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:33.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:33.353+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:33.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:33.351+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:33.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:33 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:33.351+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:33.926 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:33 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:33.501+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:33.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:33 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:33.657+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:33.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:33 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:33.657+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:34.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:34 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:34.016+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:34.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:34 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:34.016+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:34.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:33 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:33.902+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:34.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:33 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:33.902+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:34.682 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:34 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:34.286+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:34.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:34 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:34.396+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:34.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:34 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:34.396+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:34.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:34 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:34.360+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:34.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:34 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:34.360+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:34.898 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:34 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:34.490+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:34.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:34 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:34.658+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:34.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:34 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:34.658+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:35.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:34 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:34.874+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:35.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:34 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:34.874+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:35.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:35 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:35.045+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:35.265 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:35 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:35.045+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:35.694 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:35 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:35.330+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:35.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:35.406+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:35.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:35.406+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:35.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:35.406+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:35.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:35.326+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:35.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:35.326+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:35.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:35 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:35.326+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:35.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:35 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:35.463+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:35.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:35 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:35.670+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:35.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:35 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:35.670+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:36.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:36 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:36.071+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:36.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:36 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:36.071+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:36.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:35 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:35.870+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:36.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:35 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:35.870+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:36.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:36 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:36.325+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:36.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:36.453+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:36.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:36.453+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:36.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:36.453+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:36.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:36.335+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:36.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:36.335+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:36.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:36 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:36.335+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:36.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:36 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:36.421+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:37.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:36 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:36.690+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:37.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:36 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:36.690+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:37.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:36 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:36.822+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:37.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:36 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:36.822+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:37.264 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:37.022+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.470030+0000 front 2023-07-14T19:39:56.469965+0000 (oldest deadline 2023-07-14T19:40:24.670690+0000) 2023-07-14T19:40:37.265 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5[140381]: 2023-07-14T19:40:37.022+0000 7f88cda9a700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.671290+0000 front 2023-07-14T19:40:04.671318+0000 (oldest deadline 2023-07-14T19:40:29.971318+0000) 2023-07-14T19:40:37.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:37 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:37.327+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:37.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:37.455+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:37.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:37.455+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:37.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:37.455+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:37.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:37.338+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:37.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:37.338+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:37.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:37 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:37.338+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:37.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:37.418+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:37.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:37.418+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:38.195 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:37.811+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:38.195 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:37 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:37.811+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:38.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:37 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:37.705+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:38.206 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:37 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:37.705+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:38.469 INFO:journalctl@ceph.osd.5.smithi161.stdout:Jul 14 19:40:38 smithi161 bash[162591]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-5 2023-07-14T19:40:38.681 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:38 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:38.374+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:38.735 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:38 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:38.444+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:38.735 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:38 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:38.444+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:38.736 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:38.494+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:38.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:38.494+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:38.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:38.494+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:38.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:38.387+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:38.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:38.387+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:38.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:38 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:38.387+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:38.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:38 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:38.657+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:38.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:38 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:38.657+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:38.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:38 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:38.657+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:39.014 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:38 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:38.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:39.014 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:38 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:38.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:39.556 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.5.service' 2023-07-14T19:40:39.597 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:39.598 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-14T19:40:39.598 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-14T19:40:39.598 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.6 2023-07-14T19:40:39.660 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:39 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:39.375+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:39.660 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:39 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:39.375+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:39.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:39.387+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:39.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:39.387+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:39.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:39.387+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:39.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:39.498+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:39.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:39.498+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:39.738 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:39.498+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:39.738 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 systemd[1]: Stopping Ceph osd.6 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:39.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:39 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:39.410+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:39.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:39 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:39.410+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:39.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:39 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:39.636+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:39.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:39 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:39.636+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:39.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:39 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:39.636+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:40.236 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:39.940+0000 7f2fe022d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:40.237 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:39.940+0000 7f2fe022d700 -1 osd.6 99 *** Got signal Terminated *** 2023-07-14T19:40:40.237 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:39.940+0000 7f2fe022d700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:40.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:39 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:39.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:40.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:39 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:39.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:40.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:39 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:39.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:40.680 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:40 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:40.398+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:40.680 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:40 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:40.398+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:40.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:40.490+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:40.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:40.490+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:40.738 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:40.490+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:40.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:40.377+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:40.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:40.377+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:40.739 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:40 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:40.377+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:40.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:40 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:40.458+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:40.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:40 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:40.458+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:40.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:40 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:40.655+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:40.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:40 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:40.655+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:40.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:40 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:40.655+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:41.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:40 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:40.835+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:41.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:40 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:40.835+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:41.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:40 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:40.835+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:41.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:41 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:41.419+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:41.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:41 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:41.419+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:41.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:41 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:41.620+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:41.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:41 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:41.620+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:41.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:41 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:41.620+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:41.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:41.407+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:41.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:41.407+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:41.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:41.407+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:41.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:41.503+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:41.738 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:41.503+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:41.738 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:41 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:41.503+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:41.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:41 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:41.478+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:41.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:41 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:41.478+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:42.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:41 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:41.875+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:42.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:41 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:41.875+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:42.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:41 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:41.875+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:42.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:42.499+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:42.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:42.499+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:42.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:42.499+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:42.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:42.370+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:42.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:42.370+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:42.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:42 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:42.370+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:42.893 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:42.510+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:42.893 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:42.510+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:42.956 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:42.451+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:42.957 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:42.451+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:42.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:42.665+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:42.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:42.665+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:42.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:42 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:42.665+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:43.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:42.869+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:43.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:42.869+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:43.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:42 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:42.869+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:43.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:43 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:43.418+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:43.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:43 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:43.418+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:43.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:43 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:43.646+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:43.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:43 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:43.646+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:43.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:43 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:43.646+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:43.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:43 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:43.464+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:43.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:43 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:43.464+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:43.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:43 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:43.464+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:43.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:43 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:43.329+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:43.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:43 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:43.329+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:43.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:43 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:43.329+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:43.856 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:43 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:43.549+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:43.857 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:43 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:43.549+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:44.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:43 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:43.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:44.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:43 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:43.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:44.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:43 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:43.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:44.705 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:44.372+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:44.705 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:44.372+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:44.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:44 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:44.435+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:59.509961+0000 front 2023-07-14T19:39:59.510007+0000 (oldest deadline 2023-07-14T19:40:21.210073+0000) 2023-07-14T19:40:44.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:44 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:44.435+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.112408+0000 front 2023-07-14T19:40:05.112257+0000 (oldest deadline 2023-07-14T19:40:28.612265+0000) 2023-07-14T19:40:44.737 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:44 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6[151183]: 2023-07-14T19:40:44.435+0000 7f2fd881c700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.615247+0000 front 2023-07-14T19:40:13.615341+0000 (oldest deadline 2023-07-14T19:40:35.215938+0000) 2023-07-14T19:40:44.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:44 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:44.346+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:44.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:44 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:44.346+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:44.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:44 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:44.346+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:44.860 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:44.507+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:44.861 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:44.507+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:44.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:44.680+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:44.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:44.680+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:44.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:44.680+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:45.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:44.836+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:45.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:44.836+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:45.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:44.836+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:45.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:44 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:44.836+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:45.670 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:45 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:45.364+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:45.670 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:45 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:45.364+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:45.671 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:45 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:45.364+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:45.680 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:45.375+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:45.680 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:45.375+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:45.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:45 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:45.464+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:45.764 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:45 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:45.464+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:45.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:45.656+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:45.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:45.656+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:45.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:45 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:45.656+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:45.987 INFO:journalctl@ceph.osd.6.smithi082.stdout:Jul 14 19:40:45 smithi082 bash[188676]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-6 2023-07-14T19:40:46.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:45 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:45.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:46.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:45 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:45.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:46.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:45 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:45.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:46.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:45 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:45.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:46.654 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:46.367+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:46.654 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:46.367+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:46.655 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:46.367+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:46.661 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:46.378+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:46.661 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:46.378+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:46.661 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:46.378+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:46.662 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:46 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:46.378+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:46.824 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:46.511+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:46.824 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:46.511+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:46.824 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:46.511+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:46.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:46.630+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:46.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:46.630+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:46.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:46 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:46.630+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:46.998 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.6.service' 2023-07-14T19:40:47.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:47.039 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-14T19:40:47.040 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-14T19:40:47.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.7 2023-07-14T19:40:47.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:46.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:47.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:46.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:47.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:46.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:47.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:46 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:46.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:47.388 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 systemd[1]: Stopping Ceph osd.7 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:47.706 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:47.584+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:47.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:47.584+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:47.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:47.584+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:47.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:47.364+0000 7fb884ef3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:47.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:47.364+0000 7fb884ef3700 -1 osd.7 99 *** Got signal Terminated *** 2023-07-14T19:40:47.708 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:47.364+0000 7fb884ef3700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:47.708 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:47.377+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:47.708 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:47.377+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:47.708 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:47.377+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:47.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:47.427+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:47.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:47.427+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:47.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:47.427+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:47.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:47 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:47.427+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:48.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:47.552+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:48.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:47.552+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:48.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:47.552+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:48.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:47.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:48.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:47.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:48.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:47.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:48.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:47 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:47.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:48.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:48.362+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:48.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:48.362+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:48.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:48.362+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:48.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:48.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:48.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:48.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:48.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:48.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:48.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:48 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:48.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:48.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:48 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:48.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:48.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:48 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:48.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:48.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:48 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:48.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:48.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:48 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:48.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:49.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:48.547+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:49.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:48.547+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:49.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:48.547+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:49.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:48.762+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:49.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:48.762+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:49.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:48.762+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:49.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:48 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:48.762+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:49.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:49.395+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:49.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:49.395+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:49.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:49.395+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:49.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:49.544+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:49.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:49.544+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:49.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:49.544+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:49.708 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:49 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:49.544+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:49.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:49 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:49.444+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:49.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:49 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:49.444+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:49.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:49 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:49.444+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:49.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:49 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:49.444+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:49.829 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:49.536+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:49.829 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:49.536+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:49.829 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:49.536+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:50.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:49.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:50.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:49.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:50.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:49.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:50.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:49 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:49.805+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:50.706 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:50.382+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:50.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:50.382+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:50.707 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:50.382+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:50.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:50.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:50.707 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:50.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:50.708 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:50.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:50.708 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:50 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:50.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:50.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:50 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:50.406+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:50.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:50 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:50.406+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:50.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:50 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:50.406+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:50.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:50 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:50.406+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:51.020 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:50.546+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:51.021 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:50.546+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:51.021 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:50.546+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:51.021 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:50.546+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:51.021 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:50.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:51.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:50.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:51.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:50.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:51.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:50 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:50.758+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:51.639 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:51.335+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:51.640 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:51.335+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:51.640 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:51.335+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:51.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:51.446+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:51.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:51.446+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:51.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:51.446+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:51.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:51.446+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:51.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:51 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:51.446+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:51.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:51.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:51.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:51.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:51.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:51.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:51.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:51.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:51.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:51.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:52.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:51.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:52.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:51.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:52.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:51.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:52.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:51.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:52.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:51.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:52.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:51.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:52.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:51.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:52.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:51.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:52.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:51 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:51.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:52.648 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:52.319+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.720311+0000 front 2023-07-14T19:40:06.720417+0000 (oldest deadline 2023-07-14T19:40:30.220524+0000) 2023-07-14T19:40:52.649 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:52.319+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.622258+0000 front 2023-07-14T19:40:13.622158+0000 (oldest deadline 2023-07-14T19:40:38.922279+0000) 2023-07-14T19:40:52.649 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7[144776]: 2023-07-14T19:40:52.319+0000 7fb880ce9700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:22.423592+0000 front 2023-07-14T19:40:22.423664+0000 (oldest deadline 2023-07-14T19:40:45.923429+0000) 2023-07-14T19:40:52.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:52 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:52.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:52.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:52 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:52.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:52.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:52 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:52.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:52.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:52 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:52.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:52.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:52 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:52.410+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:52.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:52.624+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:52.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:52.624+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:52.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:52.624+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:52.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:52.624+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:52.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:52 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:52.624+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:53.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:52.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:53.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:52.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:53.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:52.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:53.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:52.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:53.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:52.729+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:53.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:52.729+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:53.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:52.729+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:53.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:52.729+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:53.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:52 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:52.729+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:53.456 INFO:journalctl@ceph.osd.7.smithi133.stdout:Jul 14 19:40:53 smithi133 bash[161058]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-7 2023-07-14T19:40:53.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:53 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:53.389+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:53.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:53 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:53.389+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:53.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:53 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:53.389+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:53.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:53 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:53.389+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:53.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:53 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:53.389+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:53.956 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:53.611+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:53.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:53.611+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:53.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:53.611+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:53.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:53.611+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:53.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:53 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:53.611+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:54.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:53.574+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:54.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:53.574+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:54.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:53.574+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:54.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:53.574+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:54.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:53.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:54.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:53.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:54.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:53.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:54.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:53.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:54.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:53 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:53.740+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:54.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.7.service' 2023-07-14T19:40:54.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:40:54.546 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-14T19:40:54.546 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-14T19:40:54.547 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.8 2023-07-14T19:40:54.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:54.376+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:54.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:54.376+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:54.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:54.376+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:54.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:54.376+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:54.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:54 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:54.376+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:54.919 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:54.721+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:54.920 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:54.721+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:54.920 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:54.721+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:54.920 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:54.721+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:54.920 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:54.721+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:54.920 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 systemd[1]: Stopping Ceph osd.8 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:40:54.921 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:54.921 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:54.921 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:54.921 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.599+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:54.921 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.895+0000 7f17b380f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:40:54.921 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.895+0000 7f17b380f700 -1 osd.8 99 *** Got signal Terminated *** 2023-07-14T19:40:54.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:54 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:54.597+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:54.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:54 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:54.597+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:54.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:54 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:54.597+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:54.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:54 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:54.597+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:54.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:54 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:54.597+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:55.264 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:54 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:54.895+0000 7f17b380f700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:40:55.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:55 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:55.330+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:55.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:55 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:55.330+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:55.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:55 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:55.330+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:55.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:55 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:55.330+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:55.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:55 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:55.330+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:55.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:55.562+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:55.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:55.562+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:55.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:55.562+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:55.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:55.562+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:55.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:55.562+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:56.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:55.634+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:56.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:55.634+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:56.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:55.634+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:56.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:55.634+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:56.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:55.748+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:56.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:55.748+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:56.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:55.748+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:56.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:55.748+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:56.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:55 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:55.748+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:56.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:56 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:56.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:56.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:56 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:56.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:56.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:56 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:56.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:56.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:56 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:56.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:56.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:56 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:56.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:56.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:56 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:56.531+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:56.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:56 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:56.531+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:56.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:56 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:56.531+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:56.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:56 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:56.531+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:56.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:56 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:56.531+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:57.014 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:56.744+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:57.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:56.744+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:57.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:56.744+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:57.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:56.744+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:57.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:56.744+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:57.016 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:56.613+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:57.016 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:56.613+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:57.016 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:56.613+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:57.016 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:56 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:56.613+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:57.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:57 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:57.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:57.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:57 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:57.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:57.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:57 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:57.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:57.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:57 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:57.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:57.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:57 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:57.291+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:57.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:57 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:57.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:57.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:57 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:57.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:57.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:57 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:57.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:57.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:57 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:57.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:57.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:57 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:57.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:58.014 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:57.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:58.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:57.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:58.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:57.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:58.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:57.568+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:58.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:57.773+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:58.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:57.773+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:58.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:57.773+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:58.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:57.773+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:58.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:57 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:57.773+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:58.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:58.289+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:58.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:58.289+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:58.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:58.289+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:58.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:58.289+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:58.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:58 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:58.289+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:58.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:58.472+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:58.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:58.472+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:58.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:58.472+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:58.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:58.472+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:58.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:58 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:58.472+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:59.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:58.607+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:59.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:58.607+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:59.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:58.607+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:59.015 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:58.607+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:59.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:58.812+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:40:59.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:58.812+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:40:59.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:58.812+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:40:59.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:58.812+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:40:59.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:58.812+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:40:59.017 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:58.812+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:40:59.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:59.324+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:40:59.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:59.324+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:40:59.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:59.324+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:40:59.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:59.324+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:40:59.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:40:59 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:40:59.324+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:40:59.853 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:59.573+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.596753+0000 front 2023-07-14T19:40:06.596865+0000 (oldest deadline 2023-07-14T19:40:30.696760+0000) 2023-07-14T19:40:59.853 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:59.573+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:11.797679+0000 front 2023-07-14T19:40:11.797697+0000 (oldest deadline 2023-07-14T19:40:37.097786+0000) 2023-07-14T19:40:59.854 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:59.573+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:21.099175+0000 front 2023-07-14T19:40:21.099214+0000 (oldest deadline 2023-07-14T19:40:45.799226+0000) 2023-07-14T19:40:59.854 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8[145896]: 2023-07-14T19:40:59.573+0000 7f17ade02700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.799841+0000 front 2023-07-14T19:40:25.799740+0000 (oldest deadline 2023-07-14T19:40:50.499610+0000) 2023-07-14T19:40:59.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:59 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:59.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:40:59.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:59 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:59.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:40:59.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:59 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:59.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:40:59.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:59 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:59.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:40:59.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:59 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:59.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:40:59.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:40:59 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:40:59.510+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:00.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:59.829+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:00.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:59.829+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:00.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:59.829+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:00.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:59.829+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:00.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:59.829+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:00.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:40:59 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:40:59.829+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:00.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:00.318+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:00.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:00.318+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:00.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:00.318+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:00.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:00.318+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:00.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:00.318+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:00.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:00 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:00.318+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:00.786 INFO:journalctl@ceph.osd.8.smithi161.stdout:Jul 14 19:41:00 smithi161 bash[162852]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-8 2023-07-14T19:41:00.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:00 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:00.481+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:00.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:00 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:00.481+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:00.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:00 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:00.481+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:00.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:00 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:00.481+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:00.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:00 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:00.481+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:00.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:00 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:00.481+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:01.264 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:00 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:00.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:01.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:00 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:00.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:01.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:00 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:00.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:01.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:00 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:00.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:01.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:00 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:00.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:01.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:00 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:00.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:01.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:01.278+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:01.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:01.278+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:01.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:01.278+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:01.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:01.278+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:01.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:01.278+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:01.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:01 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:01.278+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:01.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:01.512+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:01.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:01.512+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:01.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:01.512+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:01.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:01.512+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:01.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:01.512+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:01.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:01 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:01.512+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:02.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:01.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:02.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:01.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:02.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:01.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:02.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:01.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:02.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:01.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:02.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:01 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:01.799+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:02.247 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.8.service' 2023-07-14T19:41:02.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:02.289 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-14T19:41:02.289 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-14T19:41:02.289 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.9 2023-07-14T19:41:02.652 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.231+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:02.652 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.231+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:02.653 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.231+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:02.653 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.231+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:02.653 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.231+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:02.653 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.231+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:02.653 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 systemd[1]: Stopping Ceph osd.9 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:02.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:02.556+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:02.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:02.556+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:02.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:02.556+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:02.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:02.556+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:02.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:02.556+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:02.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:02 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:02.556+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:02.987 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.628+0000 7fb68a1f2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:02.988 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.628+0000 7fb68a1f2700 -1 osd.9 99 *** Got signal Terminated *** 2023-07-14T19:41:02.988 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:02 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:02.628+0000 7fb68a1f2700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:41:03.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:02.844+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:03.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:02.844+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:03.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:02.844+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:03.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:02.844+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:03.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:02.844+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:03.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:02 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:02.844+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:03.487 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:03.210+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:03.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:03.210+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:03.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:03.210+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:03.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:03.210+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:03.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:03.210+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:03.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:03 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:03.210+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:03.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:03.552+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:03.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:03.552+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:03.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:03.552+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:03.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:03.552+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:03.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:03.552+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:03.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:03 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:03.552+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:04.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:03.839+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:04.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:03.839+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:04.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:03.839+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:04.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:03.839+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:04.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:03.839+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:04.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:03.839+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:04.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:04.227+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:04.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:04.227+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:04.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:04.227+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:04.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:04.227+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:04.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:04.227+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:04.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:04 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:04.227+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:04.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:04.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:04.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:04.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:04.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:04.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:04.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:04.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:04.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:04.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:04.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:04 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:04.578+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:05.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:04.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:05.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:04.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:05.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:04.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:05.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:04.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:05.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:04.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:05.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:04 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:04.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:05.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:05.216+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:05.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:05.216+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:05.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:05.216+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:05.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:05.216+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:05.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:05.216+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:05.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:05 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:05.216+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:05.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:05.593+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:05.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:05.593+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:05.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:05.593+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:05.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:05.593+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:05.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:05.593+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:05.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:05 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:05.593+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:06.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:05.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:06.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:05.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:06.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:05.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:06.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:05.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:06.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:05.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:06.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:05 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:05.841+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:06.737 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:06 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:06.232+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:06.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:06 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:06.232+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:06.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:06 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:06.232+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:06.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:06 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:06.232+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:06.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:06 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:06.232+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:06.738 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:06 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:06.232+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:06.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:06.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:06.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:06.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:06.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:06.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:06.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:06 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:06.581+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:07.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:06.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:07.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:06.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:07.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:06.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:07.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:06.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:07.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:06.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:07.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:06 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:06.832+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:07.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:57.943090+0000 front 2023-07-14T19:39:57.943172+0000 (oldest deadline 2023-07-14T19:40:21.443234+0000) 2023-07-14T19:41:07.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:04.745562+0000 front 2023-07-14T19:40:04.745392+0000 (oldest deadline 2023-07-14T19:40:27.645249+0000) 2023-07-14T19:41:07.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.546439+0000 front 2023-07-14T19:40:13.546268+0000 (oldest deadline 2023-07-14T19:40:35.246323+0000) 2023-07-14T19:41:07.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:16.947494+0000 front 2023-07-14T19:40:16.947468+0000 (oldest deadline 2023-07-14T19:40:45.748196+0000) 2023-07-14T19:41:07.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:25.748813+0000 front 2023-07-14T19:40:25.748949+0000 (oldest deadline 2023-07-14T19:40:51.048826+0000) 2023-07-14T19:41:07.489 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:34.951264+0000 front 2023-07-14T19:40:34.951293+0000 (oldest deadline 2023-07-14T19:41:00.251151+0000) 2023-07-14T19:41:07.489 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:07 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9[156674]: 2023-07-14T19:41:07.204+0000 7fb6857e7700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:43.752153+0000 front 2023-07-14T19:40:43.752182+0000 (oldest deadline 2023-07-14T19:41:06.652160+0000) 2023-07-14T19:41:07.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:07.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:07.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:07.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:07.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:07.960 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:07.960 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:07 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:07.579+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:08.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:07.808+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:08.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:07.808+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:08.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:07.808+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:08.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:07.808+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:08.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:07.808+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:08.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:07.808+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:08.488 INFO:journalctl@ceph.osd.9.smithi082.stdout:Jul 14 19:41:08 smithi082 bash[188938]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-9 2023-07-14T19:41:08.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:08.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:08.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:08.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:08.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:08.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:08.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:08 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:08.601+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:09.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:08.776+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:09.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:08.776+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:09.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:08.776+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:09.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:08.776+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:09.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:08.777+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:09.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:08 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:08.777+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:09.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.9.service' 2023-07-14T19:41:09.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:09.822 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-14T19:41:09.823 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-14T19:41:09.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.10 2023-07-14T19:41:09.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:09.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:09.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:09.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:09.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:09.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:09.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:09.615+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:09.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:09 smithi133 systemd[1]: Stopping Ceph osd.10 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:10.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:09.807+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:10.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:09.807+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:10.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:09.807+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:10.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:09.807+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:10.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:09.807+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:10.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:09 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:09.807+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:10.457 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.143+0000 7f7205f66700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:10.457 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.143+0000 7f7205f66700 -1 osd.10 99 *** Got signal Terminated *** 2023-07-14T19:41:10.457 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.143+0000 7f7205f66700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:41:10.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:10.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:10.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:10.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:10.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:10.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:10.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:10 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:10.651+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:11.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:10.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:11.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:10.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:11.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:10.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:11.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:10.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:11.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:10.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:11.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:10 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:10.842+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:11.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:11.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:11.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:11.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:11.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:11.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:11.959 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:11 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:11.607+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:12.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:11.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:12.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:11.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:12.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:11.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:12.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:11.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:12.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:11.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:12.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:11.818+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:12.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.638+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:12.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.639+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:12.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.639+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:12.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.639+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:12.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.639+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:12.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.639+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:12.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:12 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:12.639+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:13.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:12 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:12.779+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:13.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:12 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:12.779+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:13.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:12 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:12.779+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:13.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:12 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:12.779+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:13.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:12 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:12.779+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:13.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:12 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:12.779+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:13.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:13.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:13.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:13.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:13.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:13.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:13.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:13 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:13.605+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:14.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:14.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:14.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:14.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:14.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:14.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:14.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:13 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:13.767+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:14.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:56.253287+0000 front 2023-07-14T19:39:56.253299+0000 (oldest deadline 2023-07-14T19:40:25.653702+0000) 2023-07-14T19:41:14.957 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:05.654153+0000 front 2023-07-14T19:40:05.654118+0000 (oldest deadline 2023-07-14T19:40:29.154271+0000) 2023-07-14T19:41:14.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.755847+0000 front 2023-07-14T19:40:13.755836+0000 (oldest deadline 2023-07-14T19:40:37.855697+0000) 2023-07-14T19:41:14.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:20.558336+0000 front 2023-07-14T19:40:20.558441+0000 (oldest deadline 2023-07-14T19:40:48.158786+0000) 2023-07-14T19:41:14.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:28.159402+0000 front 2023-07-14T19:40:28.159357+0000 (oldest deadline 2023-07-14T19:40:51.059423+0000) 2023-07-14T19:41:14.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.360598+0000 front 2023-07-14T19:40:36.360556+0000 (oldest deadline 2023-07-14T19:40:59.260610+0000) 2023-07-14T19:41:14.958 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:14 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10[150247]: 2023-07-14T19:41:14.561+0000 7f7200559700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.82:6822 osd.6 since back 2023-07-14T19:40:40.361625+0000 front 2023-07-14T19:40:40.361581+0000 (oldest deadline 2023-07-14T19:41:05.661503+0000) 2023-07-14T19:41:15.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:15.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:15.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:15.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:15.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:15.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:15.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:14 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:14.763+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:16.208 INFO:journalctl@ceph.osd.10.smithi133.stdout:Jul 14 19:41:15 smithi133 bash[161329]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-10 2023-07-14T19:41:16.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:16.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:16.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:16.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:16.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:16.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:16.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:15 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:15.790+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:17.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.10.service' 2023-07-14T19:41:17.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:17.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:17.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:17.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:17.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:17.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:17.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:16 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:16.793+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:17.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:17.276 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-14T19:41:17.276 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-14T19:41:17.276 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.11 2023-07-14T19:41:17.641 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 systemd[1]: Stopping Ceph osd.11 for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:18.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.616+0000 7f297fba5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:18.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.616+0000 7f297fba5700 -1 osd.11 99 *** Got signal Terminated *** 2023-07-14T19:41:18.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.616+0000 7f297fba5700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-14T19:41:18.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:18.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:18.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:18.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:18.017 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:18.017 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:18.017 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:17 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:17.769+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:19.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:19.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:19.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:19.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:19.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:19.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:19.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:18 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:18.765+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:20.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:20.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:20.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:20.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:20.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:20.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:20.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:19 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:19.734+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:21.265 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:21.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:21.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:21.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:21.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:21.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:21.266 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:20 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:20.761+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:22.015 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6806 osd.0 since back 2023-07-14T19:39:55.387983+0000 front 2023-07-14T19:39:55.388053+0000 (oldest deadline 2023-07-14T19:40:20.688061+0000) 2023-07-14T19:41:22.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6804 osd.1 since back 2023-07-14T19:40:06.490460+0000 front 2023-07-14T19:40:06.490505+0000 (oldest deadline 2023-07-14T19:40:27.590328+0000) 2023-07-14T19:41:22.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2023-07-14T19:40:13.991886+0000 front 2023-07-14T19:40:13.991915+0000 (oldest deadline 2023-07-14T19:40:39.292060+0000) 2023-07-14T19:41:22.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.82:6814 osd.3 since back 2023-07-14T19:40:19.292773+0000 front 2023-07-14T19:40:19.292783+0000 (oldest deadline 2023-07-14T19:40:44.592571+0000) 2023-07-14T19:41:22.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6812 osd.4 since back 2023-07-14T19:40:26.893619+0000 front 2023-07-14T19:40:26.893697+0000 (oldest deadline 2023-07-14T19:40:50.993605+0000) 2023-07-14T19:41:22.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2023-07-14T19:40:36.894554+0000 front 2023-07-14T19:40:36.894588+0000 (oldest deadline 2023-07-14T19:40:58.594697+0000) 2023-07-14T19:41:22.016 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:21 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11[151357]: 2023-07-14T19:41:21.736+0000 7f2978194700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.133:6820 osd.7 since back 2023-07-14T19:40:51.998054+0000 front 2023-07-14T19:40:51.997954+0000 (oldest deadline 2023-07-14T19:41:13.097813+0000) 2023-07-14T19:41:23.516 INFO:journalctl@ceph.osd.11.smithi161.stdout:Jul 14 19:41:23 smithi161 bash[163123]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-osd-11 2023-07-14T19:41:24.776 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@osd.11.service' 2023-07-14T19:41:24.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:24.817 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-14T19:41:24.817 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-14T19:41:24.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.a 2023-07-14T19:41:25.181 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:41:24 smithi082 systemd[1]: Stopping Ceph mds.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:25.488 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:41:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-a[162035]: 2023-07-14T19:41:25.155+0000 7f4e4675b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:25.488 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:41:25 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-a[162035]: 2023-07-14T19:41:25.155+0000 7f4e4675b700 -1 mds.a *** got signal Terminated *** 2023-07-14T19:41:29.736 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.a.service' 2023-07-14T19:41:29.739 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:41:29 smithi082 bash[189211]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-a 2023-07-14T19:41:29.739 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:41:29 smithi082 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.a.service: Succeeded. 2023-07-14T19:41:29.739 INFO:journalctl@ceph.mds.a.smithi082.stdout:Jul 14 19:41:29 smithi082 systemd[1]: Stopped Ceph mds.a for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:41:29.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:29.773 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-14T19:41:29.774 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-14T19:41:29.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.d 2023-07-14T19:41:30.163 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:41:29 smithi082 systemd[1]: Stopping Ceph mds.d for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:30.488 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:41:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-d[161035]: 2023-07-14T19:41:30.137+0000 7f2efb68f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:30.489 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:41:30 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-d[161035]: 2023-07-14T19:41:30.137+0000 7f2efb68f700 -1 mds.d *** got signal Terminated *** 2023-07-14T19:41:34.239 INFO:journalctl@ceph.mds.d.smithi082.stdout:Jul 14 19:41:33 smithi082 bash[189351]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-d 2023-07-14T19:41:34.253 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.d.service' 2023-07-14T19:41:34.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:34.287 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-14T19:41:34.288 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-14T19:41:34.288 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.g 2023-07-14T19:41:34.650 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:41:34 smithi082 systemd[1]: Stopping Ceph mds.g for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:34.988 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:41:34 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-g[161646]: 2023-07-14T19:41:34.624+0000 7fde3b079700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:34.989 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:41:34 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-g[161646]: 2023-07-14T19:41:34.624+0000 7fde3b079700 -1 mds.g *** got signal Terminated *** 2023-07-14T19:41:39.137 INFO:journalctl@ceph.mds.g.smithi082.stdout:Jul 14 19:41:38 smithi082 bash[189500]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-g 2023-07-14T19:41:39.142 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.g.service' 2023-07-14T19:41:39.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:39.176 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-14T19:41:39.177 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-14T19:41:39.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.j 2023-07-14T19:41:39.489 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:41:39 smithi082 systemd[1]: Stopping Ceph mds.j for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:39.988 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:41:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-j[160675]: 2023-07-14T19:41:39.546+0000 7fd16c158700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:39.989 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:41:39 smithi082 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-j[160675]: 2023-07-14T19:41:39.546+0000 7fd16c158700 -1 mds.j *** got signal Terminated *** 2023-07-14T19:41:43.989 INFO:journalctl@ceph.mds.j.smithi082.stdout:Jul 14 19:41:43 smithi082 bash[189647]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-j 2023-07-14T19:41:44.067 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.j.service' 2023-07-14T19:41:44.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:44.102 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-14T19:41:44.102 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-14T19:41:44.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.b 2023-07-14T19:41:44.447 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:41:44 smithi133 systemd[1]: Stopping Ceph mds.b for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:44.708 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:41:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: 2023-07-14T19:41:44.421+0000 7f527ba7b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:44.708 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:41:44 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b[154105]: 2023-07-14T19:41:44.421+0000 7f527ba7b700 -1 mds.b *** got signal Terminated *** 2023-07-14T19:41:46.959 INFO:journalctl@ceph.mds.b.smithi133.stdout:Jul 14 19:41:46 smithi133 bash[161616]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-b 2023-07-14T19:41:47.009 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.b.service' 2023-07-14T19:41:47.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:47.044 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-14T19:41:47.045 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-14T19:41:47.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.e 2023-07-14T19:41:47.413 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:41:47 smithi133 systemd[1]: Stopping Ceph mds.e for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:47.708 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:41:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e[153424]: 2023-07-14T19:41:47.387+0000 7f131c8be700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:47.709 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:41:47 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e[153424]: 2023-07-14T19:41:47.387+0000 7f131c8be700 -1 mds.e *** got signal Terminated *** 2023-07-14T19:41:50.958 INFO:journalctl@ceph.mds.e.smithi133.stdout:Jul 14 19:41:50 smithi133 bash[161756]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-e 2023-07-14T19:41:50.977 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.e.service' 2023-07-14T19:41:51.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:51.011 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-14T19:41:51.011 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-14T19:41:51.011 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.h 2023-07-14T19:41:51.379 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:41:51 smithi133 systemd[1]: Stopping Ceph mds.h for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:51.708 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:41:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: 2023-07-14T19:41:51.353+0000 7fa7842db700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:51.709 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:41:51 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h[153766]: 2023-07-14T19:41:51.353+0000 7fa7842db700 -1 mds.h *** got signal Terminated *** 2023-07-14T19:41:55.039 INFO:journalctl@ceph.mds.h.smithi133.stdout:Jul 14 19:41:54 smithi133 bash[161894]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-h 2023-07-14T19:41:55.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.h.service' 2023-07-14T19:41:55.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:55.080 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-14T19:41:55.080 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-14T19:41:55.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.k 2023-07-14T19:41:55.459 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:41:55 smithi133 systemd[1]: Stopping Ceph mds.k for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:55.958 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:41:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-k[153087]: 2023-07-14T19:41:55.435+0000 7f83cf9e0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:55.959 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:41:55 smithi133 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-k[153087]: 2023-07-14T19:41:55.435+0000 7f83cf9e0700 -1 mds.k *** got signal Terminated *** 2023-07-14T19:41:57.744 INFO:journalctl@ceph.mds.k.smithi133.stdout:Jul 14 19:41:57 smithi133 bash[162038]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-k 2023-07-14T19:41:57.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.k.service' 2023-07-14T19:41:57.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:41:57.910 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-14T19:41:57.910 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-14T19:41:57.910 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c 2023-07-14T19:41:58.267 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:41:57 smithi161 systemd[1]: Stopping Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:41:58.766 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:41:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: 2023-07-14T19:41:58.244+0000 7fb746d2c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:41:58.767 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:41:58 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c[159744]: 2023-07-14T19:41:58.244+0000 7fb746d2c700 -1 mds.c *** got signal Terminated *** 2023-07-14T19:42:02.831 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service' 2023-07-14T19:42:02.855 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:42:02 smithi161 bash[163388]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-c 2023-07-14T19:42:02.855 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:42:02 smithi161 systemd[1]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.c.service: Succeeded. 2023-07-14T19:42:02.855 INFO:journalctl@ceph.mds.c.smithi161.stdout:Jul 14 19:42:02 smithi161 systemd[1]: Stopped Ceph mds.c for 4f3eb35e-227b-11ee-9b33-001a4aab830c. 2023-07-14T19:42:02.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:42:02.867 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-14T19:42:02.868 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-14T19:42:02.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.f 2023-07-14T19:42:03.261 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:42:02 smithi161 systemd[1]: Stopping Ceph mds.f for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:42:03.516 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:42:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f[154624]: 2023-07-14T19:42:03.235+0000 7fae9ec60700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:42:03.517 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:42:03 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f[154624]: 2023-07-14T19:42:03.235+0000 7fae9ec60700 -1 mds.f *** got signal Terminated *** 2023-07-14T19:42:07.017 INFO:journalctl@ceph.mds.f.smithi161.stdout:Jul 14 19:42:06 smithi161 bash[163531]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-f 2023-07-14T19:42:07.159 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.f.service' 2023-07-14T19:42:07.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:42:07.194 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-14T19:42:07.194 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-14T19:42:07.195 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.i 2023-07-14T19:42:07.517 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:42:07 smithi161 systemd[1]: Stopping Ceph mds.i for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:42:08.017 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:42:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-i[155299]: 2023-07-14T19:42:07.565+0000 7f28ae082700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:42:08.017 INFO:journalctl@ceph.mds.i.smithi161.stdout:Jul 14 19:42:07 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-i[155299]: 2023-07-14T19:42:07.565+0000 7f28ae082700 -1 mds.i *** got signal Terminated *** 2023-07-14T19:42:11.129 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.i.service' 2023-07-14T19:42:11.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:42:11.163 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-14T19:42:11.163 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-14T19:42:11.164 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.l 2023-07-14T19:42:11.517 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:42:11 smithi161 systemd[1]: Stopping Ceph mds.l for 4f3eb35e-227b-11ee-9b33-001a4aab830c... 2023-07-14T19:42:12.017 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:42:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-l[154283]: 2023-07-14T19:42:11.515+0000 7f9469702700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-14T19:42:12.017 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:42:11 smithi161 ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-l[154283]: 2023-07-14T19:42:11.515+0000 7f9469702700 -1 mds.l *** got signal Terminated *** 2023-07-14T19:42:15.828 INFO:journalctl@ceph.mds.l.smithi161.stdout:Jul 14 19:42:15 smithi161 bash[163814]: ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c-mds-l 2023-07-14T19:42:15.970 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f3eb35e-227b-11ee-9b33-001a4aab830c@mds.l.service' 2023-07-14T19:42:16.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-14T19:42:16.005 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-14T19:42:16.005 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T19:42:16.306 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:42:17.863 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T19:42:18.212 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:42:19.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --force --keep-logs 2023-07-14T19:42:20.076 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:42:21.636 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:42:21.672 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:42:21.708 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-14T19:42:21.738 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-14T19:42:21.740 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi082/crash 2023-07-14T19:42:21.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T19:42:21.770 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T19:42:21.770 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2023-07-14T19:42:21.772 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi133/crash 2023-07-14T19:42:21.773 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T19:42:21.817 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T19:42:21.818 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2023-07-14T19:42:21.819 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi161/crash 2023-07-14T19:42:21.820 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash -- . 2023-07-14T19:42:21.855 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-14T19:42:21.856 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2023-07-14T19:42:21.856 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-14T19:42:21.857 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-14T19:42:21.890 INFO:teuthology.orchestra.run.smithi082.stderr:grep: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph.log: No such file or directory 2023-07-14T19:42:21.892 INFO:tasks.cephadm:Compressing logs... 2023-07-14T19:42:21.892 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T19:42:21.936 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T19:42:21.939 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-14T19:42:21.962 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T19:42:21.963 INFO:teuthology.orchestra.run.smithi082.stderr:: No such file or directory 2023-07-14T19:42:21.963 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.0.log 2023-07-14T19:42:21.963 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.g.log 2023-07-14T19:42:21.964 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.a.log 2023-07-14T19:42:21.964 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.0.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.6.log 2023-07-14T19:42:21.965 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.3.log 2023-07-14T19:42:21.965 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2023-07-14T19:42:21.966 INFO:teuthology.orchestra.run.smithi161.stderr:-5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T19:42:21.966 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.5.log 2023-07-14T19:42:21.967 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.2.log 2023-07-14T19:42:21.967 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.11.log 2023-07-14T19:42:21.967 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.5.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.z.log 2023-07-14T19:42:21.967 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.8.log 2023-07-14T19:42:21.967 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.11.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T19:42:21.969 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.f.log 2023-07-14T19:42:21.969 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.8.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.c.log 2023-07-14T19:42:21.969 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.6.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.3.log: 94.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.g.log.gz 2023-07-14T19:42:21.970 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.d.log 2023-07-14T19:42:21.970 INFO:teuthology.orchestra.run.smithi082.stderr: 94.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.a.log.gz 2023-07-14T19:42:21.970 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T19:42:21.970 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.x.log 2023-07-14T19:42:21.970 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.9.log 2023-07-14T19:42:21.970 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T19:42:21.972 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.9.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.d.log.gz 2023-07-14T19:42:21.973 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.j.log 2023-07-14T19:42:21.974 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.i.log 2023-07-14T19:42:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.j.log: 91.6% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T19:42:21.978 INFO:teuthology.orchestra.run.smithi082.stderr: 94.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.j.log.gz 2023-07-14T19:42:21.978 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.a.log 2023-07-14T19:42:21.980 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.i.log: 91.8% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.z.log.gz 2023-07-14T19:42:21.980 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.c.log 2023-07-14T19:42:21.983 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-14T19:42:21.984 INFO:teuthology.orchestra.run.smithi133.stderr:: No such file or directory 2023-07-14T19:42:21.991 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T19:42:21.993 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.10.log 2023-07-14T19:42:21.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.h.log 2023-07-14T19:42:21.996 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.e.log 2023-07-14T19:42:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.4.log 2023-07-14T19:42:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.1.log 2023-07-14T19:42:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.k.log 2023-07-14T19:42:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log 2023-07-14T19:42:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.7.log 2023-07-14T19:42:21.997 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.b.log 2023-07-14T19:42:21.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.b.log 2023-07-14T19:42:21.998 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.y.log 2023-07-14T19:42:21.998 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.b.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.b.log: /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.y.log: 94.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.k.log.gz 2023-07-14T19:42:22.003 INFO:teuthology.orchestra.run.smithi133.stderr: 91.7% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.y.log.gz 2023-07-14T19:42:22.007 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.l.log 2023-07-14T19:42:22.018 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T19:42:22.018 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.l.log.gz 2023-07-14T19:42:22.030 INFO:teuthology.orchestra.run.smithi133.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-14T19:42:22.049 INFO:teuthology.orchestra.run.smithi133.stderr: 91.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-volume.log.gz 2023-07-14T19:42:22.166 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.a.log: 89.8% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mgr.x.log.gz 2023-07-14T19:42:22.411 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.c.log.gz 2023-07-14T19:42:22.862 INFO:teuthology.orchestra.run.smithi133.stderr: 92.1% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.b.log.gz 2023-07-14T19:42:23.132 INFO:teuthology.orchestra.run.smithi082.stderr: 90.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mon.a.log.gz 2023-07-14T19:42:32.784 INFO:teuthology.orchestra.run.smithi161.stderr: 95.0% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.c.log.gz 2023-07-14T19:42:32.965 INFO:teuthology.orchestra.run.smithi161.stderr: 94.8% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.f.log.gz 2023-07-14T19:42:41.824 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.9.log.gz 2023-07-14T19:42:45.356 INFO:teuthology.orchestra.run.smithi082.stderr: 91.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.6.log.gz 2023-07-14T19:42:47.907 INFO:teuthology.orchestra.run.smithi161.stderr: 91.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.2.log.gz 2023-07-14T19:42:48.425 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.0.log.gz 2023-07-14T19:42:49.829 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.8.log.gz 2023-07-14T19:42:50.641 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.11.log.gz 2023-07-14T19:42:51.831 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.3.log.gz 2023-07-14T19:42:51.833 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-07-14T19:42:51.833 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m29.882s 2023-07-14T19:42:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:user 1m37.484s 2023-07-14T19:42:51.834 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m3.283s 2023-07-14T19:42:56.494 INFO:teuthology.orchestra.run.smithi161.stderr: 91.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.5.log.gz 2023-07-14T19:43:09.859 INFO:teuthology.orchestra.run.smithi133.stderr: 95.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.e.log.gz 2023-07-14T19:43:16.414 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.4.log.gz 2023-07-14T19:43:19.328 INFO:teuthology.orchestra.run.smithi133.stderr: 91.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.10.log.gz 2023-07-14T19:43:21.166 INFO:teuthology.orchestra.run.smithi133.stderr: 91.3% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.1.log.gz 2023-07-14T19:43:21.852 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-osd.7.log.gz 2023-07-14T19:43:52.737 INFO:teuthology.orchestra.run.smithi161.stderr: 93.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.i.log.gz 2023-07-14T19:43:52.739 INFO:teuthology.orchestra.run.smithi161.stderr: 2023-07-14T19:43:52.740 INFO:teuthology.orchestra.run.smithi161.stderr:real 1m30.782s 2023-07-14T19:43:52.740 INFO:teuthology.orchestra.run.smithi161.stderr:user 3m42.565s 2023-07-14T19:43:52.740 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m6.987s 2023-07-14T19:44:57.917 INFO:teuthology.orchestra.run.smithi133.stderr: 93.5% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.b.log.gz 2023-07-14T19:46:07.499 INFO:teuthology.orchestra.run.smithi133.stderr: 93.1% -- replaced with /var/log/ceph/4f3eb35e-227b-11ee-9b33-001a4aab830c/ceph-mds.h.log.gz 2023-07-14T19:46:07.513 INFO:teuthology.orchestra.run.smithi133.stderr: 2023-07-14T19:46:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:real 3m45.552s 2023-07-14T19:46:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:user 6m7.268s 2023-07-14T19:46:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m13.841s 2023-07-14T19:46:07.514 INFO:tasks.cephadm:Archiving logs... 2023-07-14T19:46:07.515 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi082/log 2023-07-14T19:46:07.516 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T19:46:21.175 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi133/log 2023-07-14T19:46:21.176 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T19:47:05.860 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi161/log 2023-07-14T19:47:05.861 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-14T19:47:30.761 INFO:tasks.cephadm:Removing cluster... 2023-07-14T19:47:30.762 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --force 2023-07-14T19:47:31.064 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:47:31.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --force 2023-07-14T19:47:32.554 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:47:33.437 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f3eb35e-227b-11ee-9b33-001a4aab830c --force 2023-07-14T19:47:33.737 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 4f3eb35e-227b-11ee-9b33-001a4aab830c 2023-07-14T19:47:34.482 INFO:tasks.cephadm:Removing cephadm ... 2023-07-14T19:47:34.482 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T19:47:34.504 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T19:47:34.539 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-14T19:47:34.557 INFO:tasks.cephadm:Teardown complete 2023-07-14T19:47:34.557 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-14T19:47:34.569 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-14T19:47:34.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T19:47:34.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T19:47:34.581 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-14T19:47:34.703 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-14T19:47:34.715 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-14T19:47:34.715 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T19:47:34.719 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T19:47:34.721 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-14T19:47:34.734 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-07-14T19:47:34.737 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-07-14T19:47:34.738 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-07-14T19:47:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T19:47:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-07-14T19:47:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +1083us[+1083us] +/- 87ms 2023-07-14T19:47:34.740 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -4053us[-4005us] +/- 55ms 2023-07-14T19:47:34.740 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +4011us[+4011us] +/- 67ms 2023-07-14T19:47:34.740 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:47:34.742 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T19:47:34.742 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-07-14T19:47:34.742 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +1038us[+1038us] +/- 87ms 2023-07-14T19:47:34.742 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -4105us[-4122us] +/- 55ms 2023-07-14T19:47:34.742 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +3966us[+3966us] +/- 67ms 2023-07-14T19:47:34.742 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:47:34.748 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-14T19:47:34.748 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-07-14T19:47:34.748 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +1037us[+1037us] +/- 87ms 2023-07-14T19:47:34.749 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -4124us[-4143us] +/- 55ms 2023-07-14T19:47:34.749 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +3953us[+3953us] +/- 67ms 2023-07-14T19:47:34.749 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-14T19:47:34.750 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-14T19:47:34.761 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-14T19:47:34.762 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-14T19:47:34.792 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T19:47:34.894 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T19:47:35.062 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-14T19:47:35.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T19:47:35.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:47:35.192 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T19:47:35.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:47:35.227 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-14T19:47:35.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:47:35.259 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-14T19:47:35.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-14T19:47:35.315 INFO:teuthology.task.internal:Duration was 2995.857503 seconds 2023-07-14T19:47:35.315 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-14T19:47:35.365 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-14T19:47:35.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T19:47:35.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T19:47:35.370 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-14T19:47:35.413 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T19:47:35.417 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T19:47:35.457 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-14T19:47:35.787 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-14T19:47:35.787 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2023-07-14T19:47:35.788 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T19:47:35.809 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2023-07-14T19:47:35.809 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T19:47:35.839 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2023-07-14T19:47:35.840 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-14T19:47:35.862 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-14T19:47:35.862 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T19:47:35.865 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T19:47:35.882 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-14T19:47:35.951 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-14T19:47:35.952 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T19:47:35.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T19:47:35.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-14T19:47:36.061 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-14T19:47:36.072 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-14T19:47:36.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T19:47:36.099 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T19:47:36.127 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-14T19:47:36.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-14T19:47:36.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T19:47:36.175 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T19:47:36.177 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-14T19:47:36.198 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2023-07-14T19:47:36.221 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2023-07-14T19:47:36.234 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2023-07-14T19:47:36.254 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T19:47:36.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:47:36.270 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T19:47:36.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:47:36.286 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-14T19:47:36.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-14T19:47:36.310 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-14T19:47:36.322 INFO:teuthology.task.internal:Transferring archived files... 2023-07-14T19:47:36.323 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi082 2023-07-14T19:47:36.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T19:47:36.361 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi133 2023-07-14T19:47:36.362 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T19:47:36.403 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053/remote/smithi161 2023-07-14T19:47:36.404 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-14T19:47:36.451 INFO:teuthology.task.internal:Removing archive directory... 2023-07-14T19:47:36.451 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T19:47:36.454 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T19:47:36.456 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-14T19:47:36.505 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-14T19:47:36.527 INFO:teuthology.task.internal:Not uploading archives. 2023-07-14T19:47:36.528 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-14T19:47:36.549 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-14T19:47:36.549 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T19:47:36.552 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T19:47:36.555 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-14T19:47:36.566 INFO:teuthology.orchestra.run.smithi082.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 19:47 /home/ubuntu/cephtest 2023-07-14T19:47:36.568 INFO:teuthology.orchestra.run.smithi133.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 19:47 /home/ubuntu/cephtest 2023-07-14T19:47:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 14 19:47 /home/ubuntu/cephtest 2023-07-14T19:47:36.583 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-14T19:47:36.599 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-14T19:47:36.700 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T19:47:41.726 DEBUG:teuthology.nuke:shortname: smithi082 2023-07-14T19:47:41.726 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:47:46.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:01.891886', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:47:46.764 DEBUG:teuthology.nuke:shortname: smithi133 2023-07-14T19:47:46.764 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:47:46.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:01.893602', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:47:46.792 DEBUG:teuthology.nuke:shortname: smithi161 2023-07-14T19:47:46.793 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:47:46.813 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:01.894798', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:47:46.821 INFO:teuthology.orchestra.console:Power off smithi082 2023-07-14T19:47:46.822 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:47:46.846 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:46.862 INFO:teuthology.orchestra.console:Power off smithi133 2023-07-14T19:47:46.862 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:47:46.884 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:46.901 INFO:teuthology.orchestra.console:Power off smithi161 2023-07-14T19:47:46.902 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:47:46.925 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:50.875 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:50.914 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:50.951 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:55.989 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:56.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:47:56.062 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:01.103 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:01.139 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:01.174 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:06.216 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:06.251 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:06.285 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:06.463 INFO:teuthology.orchestra.console:Power off for smithi133 completed 2023-07-14T19:48:06.500 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2023-07-14T19:48:11.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:16.444 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:21.560 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:26.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:31.787 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:36.899 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:42.011 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:47.124 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:52.236 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:48:52.449 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2023-07-14T19:48:52.554 INFO:teuthology.run:Summary data: description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/blogbench}} duration: 2995.8575026988983 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-07-14T19:48:52.555 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:48:57.655 INFO:teuthology.run:FAIL